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ABSTRACT 

The Law School Admission Council (LSAC) Skills Analysis 
Survey identifies the skills that are important for success in law school. 
This information provides validity evidence for the current Law School 
Admission Test (LSAT) and guides the development of new test items and test 
specifications. The key question of the survey is "what academic tasks are 
fundamental to success in law school courses?" The survey asked law school 
faculty and students to rate the importance of 57 law school tasks in 14 
skill-related categories as "highly," "moderately," or "somewhat" important 
or not important/not applicable for traditional first-year courses or upper 
division courses. There were 3,525 respondents from 41 law schools, of whom 
457 were faculty members. Results show that students and faculty generally 
judged the most important tasks in most of the law school courses to involve 
Reading, Analyzing Cases or Legal Problems, Reasoning, Constructing 
Arguments, Problem Solving, Time Management, Listening, and Writing. Faculty 
judged Normative Thinking Tasks to have a similar degree of importance, but 
students ranked them somewhat lower. Reasoning tasks were not judged as 
important for success in most law school courses, although they were judged 
to be important for some courses. An appendix contains the survey. (Contains 
15 figures and 3 references.) (SLD) 
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Executive Summary 

The Law School Admission Council (LSAC) Skills Analysis Survey is a major component of the LSAC 
Skills Analysis Study which in turn is an integral part of the Computerized LSAT Research Agenda. The 
goal of this survey is to identify the skills that are important for success in law school and the relationships 
among those skills. This information provides validity evidence for the current Law School Admission Test 
(LSAT) and will guide the development of new item types, item formats, and test specifications for future 
versions of the LSAT, including possible computerized versions. 

The key question the survey sets out to answer is "what academic tasks are fundamental to success in 
law school courses." The survey also asks a set of related questions about whether the ratings of the 
importance of tasks to success in courses varies importantly between subgroups of the respondents, and 
among courses, law school years, and teaching methods. If certain tasks can be identified that are 
fundamental to all or most law school courses, the skills involved in those tasks can be inferred to be 
fundamental to success in law school. 

The survey instrument asked law school faculty and students to rate the importance of 57 law school 
tasks in 14 skill-related categories as "highly important" (given the value 4), "moderately important" (3), 
"somewhat important" (2), or "not important/not applicable" (1) for one of seven traditional first-year 
courses or one of six upper-division courses. There were 3,525 respondents from 41 law schools, of which 
3,048 identified themselves as law students and 457 as faculty members. The samples of respondents and 
schools are large and are similar, in terms of geographic, gender and ethnic proportions, to the total 
populations of law students, faculty, and schools. 

For nearly all groups of respondents and for most of the courses surveyed, the categories of tasks fell 
into four tiers of importance, based on the mean ratings of the tasks and the percentage of respondents 
rating the tasks highly or moderately important. 



Tier I (mean ratings 3.4 to 4.0) 
Tier II (mean ratings 3.0 to 3.3) 



Tier III (mean ratings 2.5 to 2.9) 



Tier IV (mean ratings 1 to 2.4) 



Analyzing Cases or Legal Problems 

Problem Solving 

Reasoning 

Writing 

Reading 

Listening 

Constructing Arguments 
Managing Time 

Communicating Orally 
Organizing and Synthesizing 
Normative Thinking 

Conducting Research 

Working as Part of a Group or Interpersonally 
Quantitative Reasoning 



The survey also asked respondents about how exhaustive the list of 57 tasks was. Over 80% of the 
respondents indicated that the list either included essentially all of the tasks involved in the course, or 
included most of them. In general, faculty members considered the list to be more exhaustive than 
students did. 

These results indicate that students and faculty generally judged the most important tasks in most of the 
law school courses surveyed to involve Reading, Analyzing Cases or Legal Problems, Reasoning, 
Constructing Arguments, Problem Solving, Time Management, Listening, and Writing. Faculty also judged 
Normative Thinking tasks to have a similar degree of importance, while students rank them somewhat 
lower in importance. Organizing and Synthesizing and Communicating Orally tasks were judged less 
important for law school courses than those above, but were still judged to be of some importance. 
Conducting Research, Working as Part of a Group or Interpersonally, and Quantitative Reasoning tasks were 
not judged by the survey respondents as important for success in most law school courses, although they 
were judged to have importance in some courses. 
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Introduction 

The goal of the Law School Admission Council (LSAC) Skills Analysis Study is to identify skills 
important for success m law school and the relationships among those skills. The LSAC Skills Analysis 
.ruZI ? component of that study. Its aim is to collect and analyze the judgments of a sample of law 

•11 ° ^3culty about what tasks are important in selected law school courses. This information 

*ypes, item formats, and test specifications for the Law 

Sch(^l Admission Test (LSAT), as well as provide validation evidence for the current paper-and-pencil LSAT 

The basic testing question that lies behind the study is "what academic skills should be assesLd to 
determine a student s potential for success in law school." In other words, we are asking, "What skills are 
fundamental to success m law school?" However, answering this question directly would require law school 
faculty and students to make theoretical judgments about the relationship of skills to the academic work 
lu observed m law school. To avoid confounding the survey results with such theoretical judgments, 

the skills analysis survey instead asked about academic tasks, the identification of which should be more 
clearly a matter of empirical observation. Legal educators and test specialists can apply their expert 

judgments m a consistent way to the responses about academic tasks to infer what academic skills are 
involved in those tasks. 

To bepn to answer this question the investigators collected judgments about what tasks and skills are 
important for success in law school from (1) reports from the Association of American Law Schools (AALSI 
and the American Bar Association (ABA), (2) a telephone survey of selected law school faculty members (3) 
focus groups of students and faculty at three law schools, and (4) a report by an LSAC staff law school 
graduate. This preliminary information was used to construct a comprehensive list of law school tasks that 
included all those mentioned by these sources. This list was refined and condensed to provide the basis for a 
practical-length survey, on which law school students and faculty were asked to rate each task as "highlv 
important, moderately important," "somewhat important," or "not important/not applicable" for success 
in a particular law school course. Thus, the basic question asked by the survey is "How important is each of 
these tasks to success in this particular law school course?" These responses provide an indication of what 
tasks are considered important by law school faculty and students in one of a selected group of law school 
courses— both first-year and upper-level. ^ ^ 

In addition to the basic question of how important certain academic tasks are to success in law school 
courses a group of related questions were identified. Discussions with law school students and facultv 
revealed that they observed different methods of teaching in law school courses and that they thought this 
made a ditterence m how success in these courses was achieved. This led to the question: Does the 

importance of academic tasks to success in a law school course differ according to the methodologv used to 
teach the course? 

Focus groups with law school faculty and students also revealed differences in judgments about the 
importance of certain tasks. This led to the question: Do law school students and faculty differ in their 
judgments of the importance of tasks to success in law school courses? 

In the light of current educational literature about the different ways students and faculty of different 
pnders and ethnicities experience law school, the following questions were also asked: Do male and female 
law school students and faculty differ in their judgments of the importance of tasks to success in law school 
courses? Is there a difference in judgments about what academic tasks are important for success in law 
school courses between respondents of different race and ethnicity? Is there a difference in judgments about 
what academic tasks are important for success in law school courses between respondents whose primary 
language is English and those whose primary language is other than English? 

Traditional first-year law school courses are taught in very similar ways across courses, professors, and 
schools. In contrast, some upper-level law school courses are taught quite differently from traditional 
first-year courses and quite differently from each other. This led to the question: Are there differences 
between courses m what tasks are important for success and, specifically, are there differences between the 
tasks important for success in traditional first-year courses and those important for success in upper-level 
courses that differ from first-year courses in subject matter and teaching methods? 

Law students come from a wide variety of undergraduate backgrounds, as reflected in their 
undergraduate majors. As a result the question was posed: Is there a difference in judgments about what 
academic tasks are important for success in law school courses between students with different 
undergraduate specializations? 

Finally, respondents were asked to judge the extent to which the list of tasks in the survey includes all 
the tasks necessary for success in a particular law school course. If they thought there were tasks important 
tor the course that were not included in the survey, they were asked to list them. 
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Survey Methodology 
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Preliminary Information Gathering 

Three legal-education documents played a key role in developing the survey: (1) the 1952 AALS 
statement on prelegal education; (2) Preparation for Legal Education (1996), prepared by the Pre-Law 
Committee of the ABA Section of Legal Education and Admissions to the Bar; and (3) Legal Education and 
Professional Development— An Educational Continuum (1992), popularly known as the McCrate Report, which 
is a report of the "Task Force on Law Schools and the Profession: Narrowing the Gap," for the ABA Section 
of Legal Education and Admissions to the Bar. These documents represent considered and consensus 
judgments of legal education professionals on the skills and tasks involved in legal education and thus 
provide a solid initial basis for a survey collecting judgments of law school faculty and students. 

A second initial source of information for the survey instrument was a small, informal survey of law 
school faculty members recommended by their peers as having reflected on and studied legal education, and 
who would reflect the diversity of law schools and law professors involved with LSAC. We contacted 27 
people by telephone and gave them the following instructions: 

(1) Make a list of skills important for entering students to have for successful performance in law 
school. If you wish, you may organize your list in a way that shows their relationships and relative 
importance. 

(2) Make a list of tasks that law students are required to perform for their classes. If you wish you may 
organize them to show their relationships and relative importance. 

The directives were deliberately left somewhat general and vague so that the responses would help us to 
develop more focused questions and lists. Of the 27 people contacted, 15 sent lists. The respondents included 
9 males and 6 females; 11 whites, 2 African Americans, and one Asian American; and 9 law school faculty 
and 6 academic support staff members. 

From the lists sent by the respondents, together with the ABA and AALS statements, we compiled 
preliminary comprehensive lists of tasks involved in law school courses and skills necessary for success in 
law school. 

The third preliminary step was to add to, refine, and verify those lists through focus groups held at 
three law schools in the Philadelphia area. The law schools were selected partly to assure that there would be 
minority representation among the students and faculty in the focus groups. At each law school we met with 
separate groups of four or five students and four or five faculty. The focus groups essentially verified the 
tasks compiled from the selected faculty members and the statements by the ABA and AALS. They also 
suggested several tasks and types of tasks not found on the preliminary lists, as well as other ideas that were 
later incorporated into the design of the survey instrument. For example, focus group participants suggested 
that we not ask about tasks for law school in general, but about tasks for specific courses. They also 
suggested that we try to distinguish courses by teaching methods, since they thought the tasks involved in 
courses would vary according to the teaching methods employed by the instructor. 

We also used the focus groups to conduct a grouping exercise in which we asked faculty members to 
arrange the tasks from our list into groups that involved approximately the same underlying skill or skills. 
The data from this exercise were analyzed with a variety of hierarchical cluster analysis methods. The law 
school faculty participants exhibited a great deal of agreement about what tasks grouped together based on 
underlying skills. Moreover, the substantive content of the statistically identified clusters displayed a strong 
similarity to the categories suggested by the ABA lists and to the conceptual analyses done by LSAC staff. 
Consequently, we used essentially these categories to group the tasks conveniently for the survey. 

Based on this work, an LSAC staff member who is a law school graduate helped develop a detailed list 
of 108 tasks involved in law school classes. 

Development of the Survey Instrument 

The list of 108 tasks was used to prepare a draft survey instrument in which law school students and 
professors would rank the importance of each task for a particular law school course. 

As a trial, the survey was administered to the LSAC Test Development and Research Committee and to a 
small sample of students at several schools. In addition, it was discussed with admission professionals and 
law school faculty at the 1998 LSAC Annual Meeting and Educational Conference. On the basis of the trials 
and discussions, the list of tasks was shortened to 57 tasks in order to make survey completion easier and to 
make it more likely that recipients of the survey would take the time to complete it. Thus, the number of 
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tasks on the final survey represents an attempt to balance a concern for completeness with a desire for a 
Appen^iT"^”''"^ the survey used^in data collection is found in 

The Questions on the Survey 

The main body of the survey (Question 4) contains a tist of 57 tasks associated with work in law school 
dtvtded mto 14 categories. The tasks fall into two basic types. Some am very concreTe taskf hke anL„me 
^am questions and reading assignments. Others are more abstract, conceptually described tasks like ^ 

seemed lo™e1nformatrv'’e'aS skal?'*' ’ decision. Both kinds of tasks were included because both 

rn?in"£?"iK*' 

Staff '^e ero^uninPsT^^ natural and logical skill-related categories developed by the LSAC 

staff. The groupings from all three sources corresponded very highly. A hierarchical cluster analvsis later 

Llrhacuasks.™"'’’ lete 

the sureev^?ADMndS A° S** f “*'8“"“ into "lii* they are grouped, refer to Question 4 on 
me survey at Appendix A. The sample survey contains the complete descriptions of the tasks seen bv the 

f of tasks m the Figures and Tables contain only abbreviated names for the tasks that are closelv 
related to but do not match exactly the descriptions in the survey ^ ^ 

Question 4 of the survey uses a version of a Likert-Scale to ask law school students and facultv to rate 
hT course, each of the 57 tasks. The ratings are "highly important," "moderately iSrtant " 
somewhat important, or not important/not applicable"; four boxes corresponding to the ratings follow 
each task and the respondent is asked to check the appropriate box. The other^questions on the smvey ask 
the respondent either to check a box corresponding to an answer or to write in a response ^ 

r„'Lrb?,ft:w.t -P-^“om,y saw .be .asks “cl^a'ud 

important to success in law school, we felt that it was 

ft. ^ A faculty and students and compare their responses. Question 1 on the survev asks 

the respondent to indicate whether she or he is a faculty member or a student ® 

To determine whether the ratmg of the importance of tasks to success in law school courses might 

oTettion 2 ^nTh^^^ ^ information about how law school courses are taught, 

SeS in the foursT"" ' respondent to check a response identifying the format and teaching^ 

Since the |ist of 57 tasks rnight not fully cover the tasks necessary for success in any particular course 

Question 5 asked the respondents whether the 57 tasks listed "include essentially all the tasks involved in 

L's'kr if ^ the tasks ... but do not include some othLs " " ^cLfsoi^^^^^^^^^ 

tasks ... but not many others ...," or "include few, if any .. " nt-mue some or me 

cTiv7H*l?=.^irH open-ended questions on the survey, in response to which respondents could 
EwpH !k specific information. Regardmg the format and method of teaching of the coLse, Question 3 

Lcture sVcrefe^^^ techniques or methods u!ed in the course other thL 

^ j ^ After the question about the comprehensiveness of the list of tasks 

fn afp fh ^k ^^®P°odents to list any additional tasks involved in success in that particular course and" 

ippard w Q^^®hon 7 asked for any further comments respondents might have 

regarding the tasks required for success in the course. ^ ^ 

contained a page asking for demographic information, including gender, race or 

S iStSp nn '""fT ^"-^y -hiimize any possible 

pr. =p! ^ 1 ‘he other questions on the survey. We asked students what other 

h f ^ k^^*^ taken in their first year of law school, giving them ten choices as well as offering them the 

Sv K u. fu I ^""^^cgcaduate major. Faculty members were asked to list any other courses 

they had taught m the last two years and the school from which they had received their law degree. 

Courses Included in the Survey 

determined that the tasks required of law students could potentially vary 
gnificantly from course to course. For that reason the survey asks students and faculty to rate tasks L ^ 
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5 /rliJf than for law school in general. This allowed us to compare the responses for 

groups of courses. The LSAT has historically been validated against fi^st-year grades 
so the list of courses to 1^ surveyed includes those traditionally considered first-year courses afa significant 
number of law schools: Contracts, Torts, Criminal Law, Constitutional Law, PropWty, Qv^ProLdure LT 
Legal Research and Reasoning. To gam a broader picture of the importance of various tasks to the studv of 
law, we surveyed students and faculty regarding some upper-level courses that were likdy to be taught 
RpS^? ^ traditional core or first-year courses: Professional Responsibility, Alternative Dispute 

Resolution, any Clinical Course, an Appellate Course, Tax, and any Seminar. TheseLurses are not intended 
to be representative of the upper-level curriculum in law school, but to include courses that would more 
fully capture the range of how law school courses are taught. 

How the Surveys Were Distributed 

Because first-year law students would not have received their grades in the courses they were currentlv 
taking and, thus, would not really know what was involved in succeeding in the course, we^decided to ^ 

*e law schools to find a course or 

courses likely to be taken by all second-year students and to distribute the surveys about first-year courses to 

faw ^honri ?i"rrH law school to identify the first-year courses at that 

law school and labeled equal numbers of surveys with the name of each course. These were then randomly 

^ ° second-year students. Thus, each second-year student received a survey asking her or him to 

rate th^e importance of tasks for success in one of the first-year courses he or she had taken the preceding 

first-year courses were randomly distributed among second-year ^ 
fSS the course they were commenting on. In the unlikely case that the course 

listed at the top of the survey was one he or she had not taken, the second-year student was asked to 
substitu e a first-year course she or he had actually taken. Using this method meant that thLe was a time 
coUect^d student took the course being asked about and when the survey information was 

CO lected While this might result m some students forgetting some of what was involved in the course or 

havinrcomXted^Jhl" 7^ j^at this was outweighed by the advantage of students 

expSn^^i^ thelzo^^^^^^ a grade, and having had time to think aLut the 

described above was not available for collecting data about upper-level courses. Some of the 
courses were offered m the third year, leaving no easy way to collect data from the students after 

offered H ^ave graduated. And students surveyed about courses 

H ? second year would be scattered m a wide variety of courses in their third year, making it 

fnSL?/'' that time. For these reasons the surveys for upper-level courses were distributed 
to students while they were taking the course, but preferably towards the end of the course. While the 
students would not yet have completed the course and, thus, would not know their grade, they would 

least a year of law school and would have some perspective on how to rate the 
To caZtl " " ‘^ken. In only a few instances third-year law students were asked 

an3 m ^ r had taken. In those Lses the students were 

allowed to select the course for which to complete the survey. Thus, the course for which a student 

completed the survey depended upon either which survey she or he randomly received for first-year courses 
or which course he or she was enrolled in for upper-level courses. In neither cLe could students LTecJ a 
course, except for the small number of cases mentioned above. 

Faculty members were given surveys and asked to indicate a course they were currently teaching or 
commonly taught. Thus, faculty members did have some leeway in choosing the course for which they 
7Zht H choices were limited by the course! they were teaching or had 

among faculty m^r^btl! constrained by the distribution of bourses 



The Sample of Schools Surveyed 

invited to participate in the Skills Analysis Survey. All of those who 
agreed to participate were included m the study sample. In addition, several schools were recruited to 
participate to provide balance in the sample of schools and increase minority participation. Forty-on? 
Su?d in Tab°?i d'stnbute surveys to faculty, students, or both. The list of participating schools is 
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Despite being mostly self-selected, the total sample of law schools was found to be highly similar to the 
total population of LSAC law schools in terms of geographic distribution, size, school average LSAT score, 
school undergraduate GPA, and percentage of minority students in the school. The comparison was based 
on data gathered about matriculants during the 1996-97 application year. 

Because it was important that the survey data include the judgments of minority law students and 
faculty, we included in the sample law schools with relatively high percentages of minority students. The 
distribution of the percentages of minority students in schools in the school sample is similar to the 
distribution of the percentages of minority students in the total distribution of all LSAC member schools, 
with the sample schools having a slightly higher mean percentage than the total of LSAC member schools. 

While the geographic distribution of schools in the sample is not exactly the same as the geographic 
distribution of LSAC schools, the sample contains some schools from every official LSAC geographic region, 
including Puerto Rico and Canada, and most regions are well represented. 

Data Acquisition 

To encourage school participation, schools were offered several different options for distributing and 
collecting the surveys. Schools could choose to distribute surveys about randomly selected first-year courses 
to second-year students; surveys about upper-level courses they were currently taking to upper-level 
students; surveys about first-year and upper-level courses they were teaching or had taught to faculty 
members; or any combination of these. In addition, schools could choose (1) to both distribute and collect the 
surveys, (2) to distribute the surveys along with postage-paid envelopes, or (3) a combination of these. 
Students and faculty given surveys along with postage-paid envelopes mailed their surveys directly to 
LSAC. For each school in the sample. Table 1 contains the number of surveys sent, the number of surveys 
returned, the method of collection used, and the resulting response rate. 

Of the 41 schools, 7 only distributed surveys about first-year courses to second-year students; one only 
distributed surveys about upper-level courses to upper-level students; 4 only distributed surveys about 
first-year or upper-level courses to faculty; and 6 only distributed surveys about first-year courses to 
second-year students and to faculty. The remaining 23 schools distributed surveys to all three groups. 

Of the 13,618 surveys distributed to the schools, 3,525 were returned, which is a response rate of 26%. Of 
these, 3,048 were returned by students and 457 were returned by faculty (20 did not give their faculty or 
student status). School response rates ranged from 4.6% (21 out of 460 surveys mailed to the school were 
returned) to 88% (44 out of 50 surveys mailed to the school were returned). The largest number of surveys 
returned by any one school was 259. 

While giving schools many different options for distributing and collecting the surveys helped 
encourage school participation, it did result in the sample of respondents not being as representative as the 
sample of schools was. 



The Student and Faculty Participant Sample 

While some law schools both distributed and collected the surveys, other schools distributed the surveys 
with return envelopes, and students and faculty could choose whether to fill out and return the survey. 
Moreover, the students in classes surveyed at a particular school might not be representative of the student 
body at that school. So self-selection and convenience played a role in determining the sample of students 
and faculty. Table 2 compares the students in the survey sample with 1998-99 LSAT takers. Table 3 compares 
the faculty in the survey sample with the law faculty listed in the 1997-98 AALS Directory of Law Teachers. 

It should be noted that the demographic information from the survey and from LSAT takers is self-reported. 
We do not know how the information in the AALS Directory was acquired. 

The reported distribution of students in our sample was very similar to that of 1998-99 LSAT takers 
(Table 2). However, the percentage of reported Black/African American participants in the survey sample is 
somewhat lower than that in the test taker population, despite the fact that the sample contains responses 
from one historically black law school and that law schools with high percentages of minority students are 
slightly overrepresented in the sample. Because the survey sample is composed of second-year or higher law 
students, who are on average older than LSAT takers, as expected, the sample of survey respondents is 
slightly older than 1998-99 test takers. The gender and ethnic distributions of the faculty survey sample are 
comparable to the gender and ethnic distributions of the faculty list on the AALS 1997-98 Directory of Law 
Teachers (Table 3). 
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TABLE 2 

Comparison of skills analusi<; final 


sample and 1998-99 LSAT taker student distribution 








Female 


Male 


No Response 


Skills Final Sample 
1998-99 LSAT Takers 


50.6 

50.6 


45.7 

49.0 


3.8 

0.4 


Ethnicity 


Skills Final Sample 


1998-99 LSAT Takers 





Aboriginal Indian/ American Indian/Alaskan Native 0.8 

Asian/ Asian American/Pacific Islander 5.6 

Black/ African American 5 ] 

Latino(a)/Chicano(a)/Puerto Rican/or other Hispanic 9.1 

White/Caucasian 71 q 

Other 2 4 

No Response 5 1 



0.8 

7.5 

10.7 

7.9 

66.8 
4.2 

1.9 



Age: 


<21 


21-24 


25-39 


>39 


No Response 


Skills Final Sample 
1998-99 LSAT Takers 


.03 

64 


22.5 
53T 


66.3 
34.4 


5.5 

6J 


5.7 

02 



TABLE 3 

Comparison of skills analysis final faculty sample and 1997-98 Directory of Law Teachers. (The directory does not provide 
information about nonresponse for gender or ethnicity.) 

Female Male No Response 

Faculty in Skills Final Sample 
1997-98 Directory of Law Teachers 



35.4 57.1 7.4 

30.2 69.8 



Ethnicity 



1997-98 Directory 
Skills Final Sample of Law Teachers 



Aboriginal Indian/American Indian/Alaskan Native 
Asian/ Asian American/ Pacific Islander 
Black/ African American 

Latino(a)/Chicano(a)/Puerto Rican/or other Hispanic 

White/Caucasian 

Other 

No Response 



0.0 


0.4 


1.8 


1.8 


3.9 


7.1 


1.8 


3.1 


80.7 


86.8 


1.8 


0.0 


10.1 





Data Analysis 



Descriptive Statistics 

The survey data are in the form of demographic information, task ratings, and survey responses to 
categorical questions. The demographic information has been summarized above. For the task ratings, 
means and frequencies were calculated for the overall set of respondents. The means were also calculated for 
the respondents grouped by sex, by student or teacher status, by law school course, and by the demographic 
variables of ethnic group and primary language. Percentages were calculated for the survey questions 
concerning courses, whether the list includes all necessary tasks, and method of teaching. These results are 
displayed in figures and tables and provide a graphical way to assess the cohesiveness of the data and how 
similar or different task mean ratings are for different groups within the data. 

Inferential Statistics 

Two sets of significance tests were performed on the three group comparisons of primary interest— students 
vs. faculty, male respondents vs. female respondents, and first-year course respondents vs. upper-level course 
respondents. The first set of tests consists of analysis of variance tests to examine the group differences in the 
data— whether or not the mean rating values for different task groups differ for various respondent groupings. 
The second set consists of chi-square ) loglinear tests to see if the respondent groups rated tasks similarly 
within tasks and within respondent groups. Taken together, these two sets of analyses provide an inferential 
measure of how consistently our respondent groups rated the 57 tasks, both across tasks and within tasks. 

Analysis of variance. The multivariate analysis of variance (MANOVA) is appropriate for assessing the group 
differences in these data. The MANOVA uses demographic and group variables, such as gender, student or 
• ' er status, and course level, to predict responses on the different tasks for each of the 14 categories of tasks. 
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Separate MANOVAs could be performed for each of the three respondent groupings, but a better way is to use a 
model in which all the group values are used to simultaneously predict task category means. This allows us to 
interpret the predictive value for each respondent group while controlling for the effects of the other groups. 

The inferential statistic of interest for the MANOVAs is the Wilk's A statistic. This statistic measures the 
disparity between group means on the tasks— the larger the disparity, the smaller the A value, and the 
smaller the p-value associated with it. The p-value that is associated with A can be compared to a 
predetermined value to see if the differences within task categories are significant. However, differences that 
are significant can be of little practical use, especially when sample sizes are as large as they are here, and so 
we use a multivariate measure of practical significance, r), that is equal to 1-A. The square root of r) is 
analogous to a correlation coefficient— the larger the value, the stronger the relationship between group 
membership and task rating. A correlation of .3 is usually the smallest that is considered to be of practical 
predictive value in social science research, so an r) value that is larger than .09, the square of .3, may be a 
useful standard of practicality for these data. 

What should the predetermined p-value be for the significance tests? The norm is to use p = .05. 
However, as there are 14 tests — one for each task category — being performed for each respondent grouping, 
the p-value should be corrected so that Type I (false positive) errors can be minimized. The Bonferroni 
correction is appropriate and simple to use, because it consists of dividing .05 by the number of tests that will 
be performed. Dividing the .05 value by 14 produces .0036, and so only p-values less than this will indicate 
significance. Finally, post hoc tests can be performed on the means within each task category if necessary. A 
Tukey Least Significant Difference (LSD) test can be performed for those task groups that have significant 
p-values and large effect sizes, and the results of these tests aid in the interpretation of the r\ value. 

Chi-scjuciTe tests: Another way to view the data is to look within tasks and examine the ratings to see 
whether these respondent groups rate the same tasks in the same way. Such a process clarifies the 
within-task ratings in a way that analysis of variance, with its overall mean values, does not. A suitable 
method for this is to use a form of categorical modeling, known as log-linear modeling, in which chi-square 
significance tests are used to show if the different variables — respondent group, task, and rating — are 
independent of one another. If the chi-square statistic associated with the log-linear model has a p-value 
less than some predetermined value, then the different variables can be considered not independent. 

The calculation of the p-values here uses the same Bonferroni method as was used with the multiple 
MANOVA tests. An appropriate set of models for these data use each of the three data groupings— male/female, 
faculty/students, first-year/upper-level — separately to predict ratings within each task category to start with. 
This means there are 3 x 14, or 42 initial significance tests; The standard p-value, 0.05, divided by 42 yields a 
p-value of 0.0012. For each task set, the only one of interest is the one that assesses independence of the interaction 
of task group ratings by data grouping by task category. If the values for any task category is found significant 
at the 0.0012 level, then those tasks will be further examined one by one to see where the rating differences lie. 

Survey Results 

Question: What tasks and categories of tasks are most important for success in law school courses in general? 

Figures lA and IB summarize the responses to the survey by all respondents for all courses by task. The 
names of the tasks are abbreviated and next to the abbreviated name, in parentheses, is an abbreviation for 
the category to which the tasks belong (see the key for these abbreviations at Appendix B). For example, WR 
represents the category Writing. In the bar graphs, each bar represents a task and the proportion of the bar in 
each type of shading represents the proportion of responses for "highly important," "moderately 
important," "somewhat important," and "not important/not applicable." The number associated with each 
bar is the mean rating of that task for all respondents where highly important is valued 4; moderately 
important, 3; somewhat important, 2; and not important/not applicable, 1. 
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FIGURE la.Proportion of responses for all respondents for each task ordered by category, including mean values, for the first 
29 tasks (N = 3,525) 



Identifying academic goals (MT) 
Allocating time to tasks (MT) 

Contributing to group (GR) 
Engaging in negotiation (GR) 

Identifying strongest point (AR) 
Organizing evidence (AR) 
Effectively countering (AR) 
Interpreting incomplete facts (AR) 

Identifying key lecture points (LI) 
Distinguishing what was said (LI) 
Identifying what was implicitly said (LI) 
Raising appropriate questions (LI) 

Answering basic class questions (OR) 
Answering other class questions (OR) 
Orally arguing persuasively (OR) 
Communicating with clients (OR) 

Outlining a course for exams(OS) 
Identifying general law theories (OS) 
Formulating hypotheses (OS) 

Writing class notes (WR) 
Writing briefs (WR) 
Writing exam answers (WR) 
Drafting legal documents (WR) 
Writing quickly (WR) 
Writing concisely (WR) 
Listing facts (WR) 
Arguing logically in writing (WR) 
Using proper grammar (WR) 
Editing (WR) 
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Mean Values 

3.1 

3.2 

2.2 

1.9 

3.3 

3.3 

3.2 

2.9 



3.4 

3.1 

3.0 

3.1 

3.0 
2.7 

2.7 

2.1 

3.0 

2.9 

2.7 



Highly 



lo:l Moderately 



Somewhat 



XJ Not 



3.1 

2.6 

3.1 

2.1 
3.0 
3.5 
3.0 

3.4 
3.0 

2.5 
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FIGURE lb. Proportion of responses for all respondents for each task ordered by category, including mean values, for the last 
28 tasks (N = 3,525) 



Deducing a legal conclusion (RS) 
Generalizing case principles (RS) 
Identifying reasoning fallacies (RS) 
Comparing different reasonings (RS) 



Identifying legal problem (PS) 
Identifying legal alternatives (PS) 

Identifying key legal facts (AC) 
Identifying legal basis (AC) 
Interpreting statutes (AC) 
Identifying rule of law (AC) 
Identifying legal arguments (AC) 
Identifying legal similarities (AC) 
Applying legal statutes (AC) 



Identifying ethical issues (NT) 
Engaging in moral reasoning (NT) 
Comparing ethical principles (NT) 



Using law library (CR) 
Using electronic databases (CR) 



Reading assigned material (RD) 

Reading class notes (RD) 

Reading supplemental material (RD) 

Reading large amounts (RD) 

Reading critically (RD) 

Making calculations (QR) 

Comparing quantities (QR) 

Interpreting formulas (QR) 

Interpreting statistics (QR) 

Interpreting graphs (QR) 

0 10 20 30 40 50 60 70 80 90 100 




Mean Values 
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3.0 

3.6 

3.5 

3.5 

3.6 
3.5 
3.4 




3.2 

2.5 

2.7 

3.3 

1.8 

1.7 

1.8 

1.6 
1.5 



Percentage of Total Responses 



The most important tasks and categories of tasks for success in law school, in the judgments of the 
respondents, will be those with the highest mean scores and those rated highly important or moderately 
important by the most respondents. One way to rank the tasks and categories is to place them in tiers 
corresponding to levels of mean ratings of importance. All of the tasks with mean ratings of 2.5 or above can 
reasonably be said to have some degree of importance in law school courses generally. The following tiers 
Q represent what appear to be groupings of importance inherent in the data, as indicated in Figure 2: 
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Tier I (Mean rating of 3.4 or higher): Highly Important 



(1) All seven Analyzing Cases or Legal Problems tasks 

■ Identifying key legal facts and issues (3.6) 

• Identifying a principle or rule of law operating in a case (3.6) 

■ Applying a case, rule or legal statute to a hypothetical case (3.6) 

■ Identifying the basis for a legal decision (3.5) 

■ Interpreting statutes in relation to a case or problem (3.5) 

■ Identifying and evaluating the legal arguments in a case (3.5) 

■ Identifying similarities and differences between cases and problems (3.4) 

(2) Two of ten Writing tasks 

• Writing concisely and with clarity (3.5) 

• Arguing logically and persuasively (3.4) 

(3) Two of four Reasoning tasks 

■ Deducing a legal conclusion (3.5) 

■ Generalizing or synthesizing principles or rules from cases (3.4) 

(4) One of two Problem Solving tasks 

■ Identifying a legal problem and the legal issues involved (3.6) 

(5) One of five Reading tasks 

• Reading assigned materials (3.5) 

(6) One of four Listening tasks 

■ Identifying key points in lectures and discussions (3.4) 

Tier II (Mean rating of 3.0 to 3.3): Important 

(1) Both Managing Time tasks 

■ Allocating time to tasks based on priorities (3.2) 

■ Identifying goals, priorities, and the tasks necessary to them (3.1) 

(2) Three of four Constructing Arguments tasks 

■ Identifying the strongest evidence or reasons for or against a position (3.3) 

■ Organizing evidence and reasons into a logical argument (3.3) 

■ Countering or rebutting evidence or arguments against a position (3.2) 

(3) Three of four Listening tasks 

■ Distinguishing precisely what was said and not said (3.1) 

■ Raising appropriate questions and arguments in response (3.1) 

■ Identifying what is implicit in what was said (3.0) 

(4) One of four Communicating Orally tasks 

■ Answering basic class questions about assigned readings (3.0) 

(5) One of three Organizing and Synthesizing tasks 

■ Outlining a course for final exams (3.0) 

(6) Five of ten Writing tasks 

■ Writing class notes (3.1) 

■ Writing answers to exam questions (3.1) 

■ Writing quickly and fluently under time pressure (3.0) 

■ Listing or describing facts (3.0) 

■ Using proper grammar, punctuation, and spelling (3.0) 

(7) Two of four Reasoning tasks 

■ Identifying and comparing patterns of reasoning (3.2) 

■ Identifying contradictions and flawed reasoning (3.1) 

(8) One of two Problem Solving tasks 

■ Identifying alternatives and developing a plan of action (3.0) 

(9) Two of five Reading tasks 

• Reading critically and interactively (3.3) 

• Reading class notes (3.2) 
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Tier III (2.5 to 2.9): Somewhat Important 

(1) One of four Constructing Arguments tasks 

■ Interpreting or shading incomplete information to support a conclusion (2.9) 

(2) Two of three Organizing and Synthesizing tasks 

■ Identifying and assessing general theories that unify areas of law (2.9) 

■ Formulating hypotheses to account for information (2.7) 

(3) Two of four Communicating Orally tasks 

■ Answering questions about unassigned hypothetical cases (2.7) 

■ Orally arguing persuasively (2.7) 

(4) Two of ten Writing tasks 

■ Writing briefs (2.6) 

■ Editing (2.5) 

(4) All three Normative Thinking tasks 

■ Identifying ethical and policy issues (2.9) 

■ Engaging in reasoning involving moral or policy principles (2.9) 

■ Comparing the weight of competing ethical or policy considerations (2.8) 

(5) Two of the five Reading tasks 

■ Reading large amounts of material under time constraints (2.7) 

■ Reading supplemental materials (2.5) 

Tier IV (below 2.5) 

(1) One of four Communicating Orally tasks 

■ Communicating orally with a client (2.1) 

(2) One of ten Writing tasks 

■ Drafting a legal document (2.1) 

(3) Both of the Conducting Research tasks 

■ Using the law library and references (2.3) 

■ Using electronic databases (2.1) 

(4) Both of the Working as Part of a Group or Interpersonally tasks 

■ Contributing to a group assignment or discussion group (2.2) 

■ Engaging in negotiation, conflict resolution, or arbitration (1.9) 

(5) All five of the Quantitative Reasoning tasks 

■ Making calculations (1.8) 

■ Interpreting and applying formulas (1.8) 

■ Comparing quantities (1.7) 

■ Interpreting statistics (1.6) 

■ Interpreting graphical representations (1.5) 

The skill-related categories of tasks also can be ranked in the same four tiers based on the importance of 
their members, with Tier I categories having tasks predominantly in Tier I, Tier II categories having tasks 
predominantly in Tiers II and I, Tier III categories having tasks predominantly in Tiers III and II, and Tier IV 
categories having tasks predominantly in Tier IV. The fit of these categories to the responses can be seen in 
Figure 2. 

Tier I: Analyzing Cases or Legal Problems 

Tier II: Problem Solving, Reasoning, Writing, Reading, Listening, Constructing Arguments, and 
Managing Time 

Tier III: Communicating Orally, Organizing and Synthesizing, and Normative Thinking 

Tier IV: Conducting Research, Working as Part of a Group or Interpersonally, and Quantitative 
Reasoning 
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The proportions of respondents rating the tasks as highly or moderately important also support these 
rankings of tasks and categories. 

50% or More of Respondents Rated Highly Important 

7 of 7 Analyzing Cases or Legal Problems tasks 

3 of 5 Reading tasks 

3 of 10 Writing tasks 

2 of 4 Reasoning tasks 

2 of 4 Constructing Argument tasks 

1 of 2 Problem Solving tasks 

1 of 3 Organizing and Synthesizing tasks 

1 of 4 Listening tasks 

50% or More of Respondents Rated Moderately Important or Higher 

7 of 7 Analyzing Cases or Legal Problems tasks 

4 of 5 Reading tasks 

8 of 10 Writing tasks 

4 of 4 Reasoning tasks 

4 of 4 Constructing Argument tasks 

2 of 2 Problem Solving tasks 

3 of 3 Organizing and Synthesizing tasks 

4 of 4 Listening tasks 

3 of 4 Communicating Orally tasks 

2 of 2 Managing Time tasks 

3 of 3 Normative Thinking tasks 

The category of tasks clearly ranked as most important is Analyzing Cases or Legal Problems. The mean 
ratings of all of the tasks in this category are 3.4 or above. The skills that directly apply to these tasks are 
taught in law school and are not expected of entering law students. However, the basic skills of critical 
reading, analyzing texts and arguments, and reasoning are the foundations of the legal analysis skills taught 
in law school and these basic skills are among those the current version of the LSAT seeks to assess. 

The second tier of categories includes Reasoning, Problem Solving, Reading, Constructing Arguments, 
Managing Time, Writing, and Listening. Reasoning, Problem Solving, Constructing Arguments, and Reading 
all involve skills that the current LSAT seeks to assess. While the current LSAT requires that test takers 
manage time over a short term, it does not test the kind of time management required of a student over an 
entire course, and it is not clear how that could be measured on a single-day standardized test. Two other 
categories of tasks—Writing and Listening— involve skills that are not currently assessed by the LSAT. 

The ratings of importance of the tasks in these categories in the survey suggest that the skills involved in 
these categories of tasks are also fundamental to success in law school and that LSAC ought to consider 
testing them. 

Two of the Reading tasks and three of the Writing tasks were rated a good deal lower than the other 
tasks m the second tier. However, most of these tasks— reading large amounts, reading supplemental 
materials, writing briefs, editing, and drafting legal documents— are highly specific tasks that do not appear 
to be important in most law school courses, even though other Reading and Writing tasks are very 
important. Therefore, we do not believe that they detract from the importance of reading and writing 
skills generally. 

Since tasks with a mean score of 2.5 or above can appropriately be judged to have been identified by 
respondents as having some degree of importance, the categories in the third tier— Organizing and 
Synthesizing, Communicating Orally, and Normative Thinking— include tasks that also are important to 
success in law school courses, although not to the extent or not so generally as the tasks in the higher tiers. 

Of the categories in the third tier. Organizing and Synthesizing skills are tested in different ways by each of 
the three item types on the current LSAT. The current LSAT also contains Logical Reasoning and Reading 
Comprehension passages that discuss ethical and public policy issues. In this way, some of the skills 
involved in the Normative Thinking category of tasks are indirectly tested. Skills in the Communicating 
Orally category are not assessed on the LSAT, since they cannot easily be tested in a paper and pencil format. 
However, the survey suggests that these skills are important in law school courses and law schools may 
want to consider ways to gather information about them. 
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Three categories of tasks — Conducting Research, Working as Part of a Group or Interpersonally, and 
QuantitaHve Reasoning— were rated as considerably less important than the others on the list, with most 
mean ratings below 2.3. All of the tasks in these categories were rated as highly important by less than 30% 
of the respondents and as not important by more than 40%. While the breakdown of the survey results for 
individual courses and the answers to the open-ended questions indicate that some of these tasks are 
important in some law school courses, the overall survey results indicate that the skills involved in these 
tasks are probably not fundamental to success in law school generally. 

Question: Do faculty and students rate the importance of tasks for law school courses differently? 

Figures 3a and 3b compare the task mean ratings of faculty and students for the 57 tasks. The tasks are 
listed along the horizontal axis and the ratings on the vertical axis. The boxes represent the responses of 
faculty and the dots the responses of students. 



FIGURE 3a. Comparison of student and faculty task mean ratings for the first 29 tasks (N - 3,505) 
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FIGURE 3b. Comparison of student and faculty task mean ratings for the last 28 tasks (N = 3,505) 




Tasks 



These figures show that the mean ratings of the tasks by students and faculty are very similar, with 
faculty generally rating the tasks as more important than students do, but with the pattern of relative ratines 
of both groups being similar. Tasks rated higher by faculty were generally rated higher by students and 
those rated lower by faculty were generally rated lower by students. The ratings of tasks in the categories of 
Managing Time, Working as Part of a Group or Interpersonally, and Conducting Research are very similar 
for faculty and ^udents. However, faculty respondents generally rated Analyzing Cases or Legal Problems, 
Listening, Oral Communication, Constructing Arguments, Reasoning, and Normative Thinking tasks as 
more important than did students— in some cases significantly more important. These differences are 
especially large for Normative Thinking tasks and for some of the Reasoning and Constructing 
Arguments tasks. ° 



Because there are so many more student responses than faculty responses in the survey, the task mean 
ratings for all respondents are very heavily influenced by the student responses. Since faculty rated many tasks 
and groups of tasks as more highly important than students did, we also looked at how the tasks and 
categories of tasks would be ranked differently if the faculty task mean ratings were used, rather than the 
studerit task mean ratings. Table 4 compares the task mean ratings for all respondents, faculty, and students 
for tasks that faculty ratings placed in a higher tier than student ratings and the ratings of all respondents did. 
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TABLE 4 

Comvarison of task mean ratings for faculty, students, and all respondents for 


the tasks faculty ranked higher than students did 




Task Mean Ratines and Tiers 


Tasks 


Faculty 


Students 


All 


Constructing Arguments 

Identifying the strongest evidence or reasons for or against a position 
Organizing evidence and reasons into a logical Argument 
Countering or rebutting evidence or arguments against a position 
Interpreting or shading incomplete information to support a conclusion 


3.6, Tier 1 
3.6, Tier 1 
3.5, Tier 1 
3.1, Tier 11 


3.3, Tier 11 
3.3, Tier 11 
3.1, Tier 11 
2.9, Tier 111 


3.3, Tier 11 
3.3, Tier 11 
3.2, Tier 11 
2.9, Tier 111 


Reasoning 

Identifying contradictions and flawed reasoning 
Identifying and comparing patterns of reasoning 


3.5, Tier 1 
3.4, Tier 1 


3.1, Tier 11 

3.2, Tier 11 


3.1, Tier 11 

3.2, Tier 11 


Normative Thinking 

Engaging in reasoning involving moral or policy principles 
Identifying ethical and policy issues 

Comparing the weight of competing ethical or policy considerations 


3.4, Tier 1 
3.3, Tier 11 
3.2, Tier 11 


2.9, Tier 111 
2.8, Tier 111 
2.8, Tier 111 


2.9, Tier 111 
2.9, Tier 111 
2.8, Tier 111 


Listening 

Raising appropriate questions and arguments in response 


3.4, Tier I 


3.0, Tier 11 


3.1, Tier 11 


Communicating Orally 

Orally arguing persuasively 


3.0, Tier II 


2.7, Tier 111 


2.7, Tier 111 


Reading 

Reading criticallv 


3.6. Tier 1 


3.3. Tier 11 


3.3. Tier II 



There are three categories of tasks in which faculty rated a significant number of tasks in a different tier 
than students did. Faculty rated all three Normative Thinking tasks as considerably more important than 
students did, suggesting that faculty consider ethical and policy reasoning to be more important in the 
courses than students do. Two Normative Thinking tasks — engaging in moral reasoning and identifying 
ethical issues — were rated as highly important by 50% or more of the faculty, whereas none of the Normative 
Thinking tasks were identified as highly important by 50% or more of the students. And the mean ratings of 
the faculty for one of the three Normative Thinking tasks place it in Tier I, and for two of the tasks, place 
them in Tier II, whereas the mean ratings of both students and all respondents place them in Tier III. Faculty 
also placed Constructing Arguments tasks in different tiers than students did. Faculty ratings placed three of 
the four Constructing Arguments tasks in Tier I and the fourth in Tier II, whereas students ratings placed 
three of the four Constructing Arguments tasks in Tier II and one in Tier III. For faculty the category 
Constructing Arguments falls in Tier I, rather than Tier II, where it falls when ranked for all respondents. In 
addition, faculty ranked all four of the Reasoning tasks in Tier I, while students ranked two in Tier I and two in 
Tier II. Thus, for faculty, the Reasoning category also falls in Tier I, whereas it falls in Tier II when ranked for 
all respondents. The rankings of categories of tasks for faculty and for all respondents are compared below: 

Faculty All Respondents 



Tier I 

Analyzing Cases or Legal Problems 
Reasoning 

Constructing Arguments 
Tier II 

Problem Solving 

Writing 

Reading 

Listening 

Managing Time 

Normative Thinking 

Tier III 

Communicating Orally 
Organizing and Synthesizing 



Tier I 

Analyzing Cases or Legal Problems 

Tier II 
Reasoning 

Constructing Arguments 

Problem Solving 

Writing 

Reading 

Listening 

Managing Time 

Tier III 

Normative Thinking 
Communicating Orally 
Organizing and Synthesizing 
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Tier IV 

Conducting Research 

Working as Part of a Group/Interpersonally 
Quantitative Reasoning 



Tier IV 

Conducting Research 

Working as Part of a Group/Interpersonally 
Quantitative Reasoning 



Faculty also rated several individual tasks in different tiers than students did. These include "raising 
appropriate questions arid arguments in response" in the Listening category, "orally arguing persuasively" in 
the Communicating Orally category, and "reading critically" in the Reading category. These differences in 
rankmgs of tasks do not affect the overall rankings of these categories for faculty, but it is noteworthy that 
faculty generally rated Listenmg and Communicating Orally tasks higher than students did. 

Students did rate some tasks as more important than faculty did, including some which they placed in a 
ditterent tier. Faculty ratings ranked "outlining a course for final exams" (OS), "writing class notes" (WR) 
'writing answers to exam questions" (WR), "listing or describing facts" (WR), and "reading class notes" (RD) in 
Tier III, whereas student ratings placed them in Tier II. Faculty ratings placed "reading large amounts" (RD) in 
Tier IV, whereas student ratings placed it in Ter III. However, in all these cases, the differences in the ratines were 
small and the overall rankmgs matched those of the students. Students also rated all four Quantitative Reasoning 
tasks soinewhat higher than faculty did, although they feU in Tier IV for both groups. It is noteworthy that most 
ot the tasks rated as more important by students are concrete, practical tasks oriented toward preparing for the 
fmal exammation, while the tasks considered more important by faculty tend to be more abstract and conceptual 
m nature. This inirrors comments made in the student and faculty focus groups. It is also noteworthy that 
students placed higher importance than faculty did on writing quickly and reading large amounts. This may 
mdicate that faculty and students have different perceptions of the workloads assigned in courses. 

Despite these real differences between the ratings of faculty and students, the overall pattern of relative 
ratings of the tasks by faculty and students are very similar. This suggests that faculty and students share 
similar conceptions of the tasks involved in law school courses and what is necessary for success. Both 
rated the same tasks as important in some respect, and the differences between the two are 
differences m degree of importance, not in whether certain tasks are important or not. Thus, the differences 
m ratings between the two groups probably don't have practical import for test validity or design 
The results of the MANOVAs are provided in Table 5. This table provides the p-values for the 
student/teai^er comparison, the male/female comparison, and the first-year/upper-level comparison. Also 
provided m Table 5 are the t| values that represent the 1-A values generated for the significance tests These 
results supplement the observations regarding mean differences that are listed above 



TABLE 5 



Ilie MANOVA result s for the 3 respondent Qroupin^s and the 14 task categories 



N umber Faculty vs. Students 

Task Group of tasks p~va\ue n value 



Managing Time 


2 




0.3272 


t| VdiUC 
0.001 


Working as Part of a Group 


2 


0.2482 


0.001 


Constructing Arguments 


4 


0.0001 


0.015 


Listening 


4 


0.0001 


0.020 


Communicating Orally 


4 


0.0001 


0.013 


Organizing and Synthesizing 


3 


0.0001 


0.019 


Writing 


10 


0.0001 


0.034 


Reasoning 


4 


0.0001 


0.031 


Problem Solving 


2 


0.0001 


0.006 


Analyzing Cases or Legal Problems 


7 


0.0001 


0.022 


Normative Thinking 


3 


0.0001 


0.031 


Conducting Research 


2 


0.0379 


0.002 


Reading 


5 


0.0001 


0.091 


Quantitative Reasoning 


5 


0.0001 


0.011 



Note. Significant p-values are in bold print. Only p-values that are significant 



_ Males vs. Females 1st Year vs. Upper Level 

p-value n value ;?-value n value 



0.1049 


0.002 


0.0001 


0.034 


0.0006 


0.005 


0.0001 


0.015 


0.4547 


0.002 


0.0001 


0.040 


0.0935 


0.003 


0.0001 


0.036 


0.0062 


0.005 


0.0001 


0.033 


0.0032 


0.005 


0.0001 


0.098 


0.0001 


0.031 


0.0001 


0.225 


0.2034 


0.002 


0.0001 


0.104 


0.4402 


0.001 


0.0001 


0.040 


0.1832 


0.004 


0.0001 


0.156 


0.0029 


0.005 


0.0846 


0.002 


0.2938 


0.001 


0.6632 


0.001 


0.0333 

0.0001 


0.004 

0.010 


0.0001 

0.0001 


0.137 

0.083 



at the Bonferroni-adjusted value of .0036 are significant. 



, , categories, 11 show significant differences between student and faculty respondents. Only the 

Managing Time, Workmg as Part of a Group or Interpersonally, and Conducting Research categories did not 
show a difference between the students and faculty. However, these differences, while significant, are mostly not 
large m a practical sense. The effect sizes are very low— the largest significant value, .091, is for the Reading task 
category, while the smallest value, .006, is for the Problem Solving category. These t| values are consistent with the 
conclusions drawn from the observations of the mean graphs in Figures 3a and 3b described above. 

The results of the log-linear analyses are shown in Table 6. The x' and p-values associated with the three 
data groupings are presented for each task category. For the eight task categories with nonsignificant values, 
the faculty and students are responding similarly as far as rating the tasks within a category. There are six 

^ significant X values for faculty vs. students task categories— Constructing Arguments, Communicating 

o 
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Orally, Organizing and Synthesizing, Writing, Reasoning, and Reading— with the largest values being for the 
Writing and Reading categories. This is entirely consistent with the observation above that students tended 
to rate concrete, practical reading and writing tasks higher than did faculty members, while also being 
consistent with the conclusion that there are no large overall differences between the faculty and students 
when it comes to rating tasks. 



TABLE 6 

The y~ lest results for the 3 respondent groupings and 14 task categories 



Task GrouD 


Number 


Facultv vs. 


Students 


Males vs. 


Females 


1st Year vs. 


Upper Level 


of tasks 


zi 


p-value 




p-value 


y' 


t?-value 


Managing Time 


2 


0.96 


0.8120 


0.45 


0.9294 


0.15 


0.9854 


Working as Part of a Group 


2 


3.30 


0.3472 


1.12 


0.7720 


11.98 


0.0074 


Constructing Arguments 


4 


29.27 


0.0006 


10.74 


0.2942 


32.40 


0.0002 


Listening 


4 


24.47 


0.0036 


11.81 


0.2241 


85.13 


0.0000 


Communicating Orally 


4 


40.10 


0.0000 


6.92 


0.6450 


142.83 


0.0000 


Organizing and Synthesizing 


3 


46.81 


0.0000 


3.53 


0.7406 


30.69 


0.0000 


Writing 


10 


103.29 


0.0000 


84.97 


0.0000 


571.86 


0.0000 


Reasoning 


4 


29.59 


0.0005 


11.86 


0.2214 


53.78 


0.0000 


Problem Solving 


2 


1.59 


0.6610 


3.48 


0.3233 


88.73 


0.0000 


Analyzing Cases or Legal Problems 


7 


34.60 


0.0106 


13.38 


0.7685 


124.76 


0.0000 


Normative Thinking 


3 


3.84 


0.6989 


6.73 


0.3465 


1.04 


0.9839 


Conducting Research 


2 


1.03 


0.7947 


1.63 


0.6522 


0.24 


0.9709 


Reading 


5 


197.43 


0.0000 


24.20 


0.0191 


125.85 


0.0000 


Ouantitative Reasoning 


5_ 


10 06 


0.6103 


8.37 


0.7555 


87,27 


0.0000 



Note. Significant p-values are in bold print. Only p-values that are significant at the Bonferroni-adjusted value of .0012 are significant. 



Question: Do men and women rate the importance of tasks for law school courses differently? 

Faculty 

Figures 4a and 4b compare the mean ratings of the importance of tasks by male and female faculty 
members. The dots represent the responses of male participants and the boxes represent those of 
female participants. 

There are some differences in the mean ratings of the importance of tasks by male and female faculty. 
The mean ratings of female faculty are slightly higher for Managing Time, Working as Part of a Group or 
Interpersonally, Conducting Research, and for some Writing tasks. For example, male faculty rated writing 
exam questions as more important than female faculty did while female faculty rated drafting legal 
documents as more important than male faculty did. In addition, female faculty rated writing mechanics 
tasks, such as editing and using proper grammar, as slightly more important than male faculty did. 

Other than these small differences, the ratings of the importance of tasks by male and female faculty 
members do not seem to differ much. It is possible that these small differences in ratings by male and female 
faculty members may be due to differences in the courses for which they responded. Figure 5 shows the 
proportion of males and female responding for each of the courses in the survey. 
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FIGURE 4a. Comparison of male and female faculty task mean ratings for the first 29 tasks (N = 455) 




Tasks 



o 

ERIC 



FIGURE 4b. Comparison of male and female faculty task mean ratings for the last 28 tasks (N = 455) 
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FIGURE 5. Proportion of male and female faculty responding for each course surveyed (N = 457) 
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Students 

Figures 6a and 6b compare the mean ratings for male and female students. These mean ratings are 
virtually identical except for male students rating Quantitative Reasoning tasks as slightly more important 
than female students did. This difference may simply be due to the proportion of male versus female 
students responding for the Tax course. 
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FIGURE 6a. Comparison of male and female student task mean ratings for the first 29 tasks (N = 3,048) 
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FIGURE 6b. Comparison of male and female student task mean ratings for the last 28 tasks (N = 3,048) 
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All Respondents 

The male-female MANOVA comparisons are provided in Table 5. There are few significant differences 
between male and female survey respondents in terms of how they rate the tasks in categories. Significant 
differences appear for the Writing and Quantitative Reasoning task categories, which is in line with 
conventional wisdom and the conclusions above. Significant differences also appear for the Working as Part 
of a Group or Interpersonally, Organizing and Synthesizing, and Normative Thinking task categories. 
However, the effect sizes for the significant differences are very small, ranging from .005 to .031, which 
indicates that these differences are probably significant only because of the large sample size. 

Table 6 shows that there is only one significant value — for the Writing task category — for males vs. 
females, indicating that male and female respondents tend to rate all of the tasks within the other 13 task 
categories similarly to one another. This may be related to the observation made above that male and female 
faculty members placed different weights on writing mechanics tasks. Overall, however, the data here 
appear even more convergent than for the faculty-student comparisons, and there do not appear to be any 
large differences in the way male and female respondents rated the tasks. 

Question: Is there a difference in the task mean ratings between traditional first-year courses collectively 
and the upper-level courses collectively? 

Figures 7a and 7b compare the mean ratings of the importance of tasks for traditional first-year courses 
collectively with those of the importance of tasks for the upper-level courses surveyed collectively. The dots 
indicate the responses for first-year courses and the boxes indicate the responses for upper-level courses. The 
responses for students and faculty were very comparable for this comparison and so were aggregated. 



FIGURE 7a. Comparison of task mean ratings for first-year courses collectively vs. upper-level courses collectively 
for the first 29 tasks (N - 3,520) 
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fo^th^k^8^tash‘^[N°- ratings for first-year courses collectively vs. upper-level courses collectively 
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For the tasks in the Normative Thinking category, the Conducting Research category, and for three of the 
Listening tasks, there were no apparent differences between the mean ratings of their importance for 
traditional first-year courses and for the upper-level courses surveyed. The tasks in seven of the 
categories— Managing Time, Constructing Arguments, Organizing and Synthesizing, Writing, Reasoning, 
Analyzing Legal Cases, and Reading— were rated as more important for traditional first-year courses than 
for the upper-level courses surveyed. The tasks in two of the categories— Working as Part of a Group or 
Interpersonally and Quantitative Reasoning — were rated as more important for upper-level courses than for 
the traditional first-year courses. This is consistent with the nature of the upper-level courses selected for the 
survey, which included Clinical courses. Alternative Dispute Resolution, and Tax. 

The ratings for tasks in the Communicating Orally and Problem Solving categories varied between being 
more important for the upper-level courses surveyed and for traditional first-year courses. For example, in 
the Communicating Orally category, the two tasks associated with answering questions had higher mean 
ratings of importance for traditional first-year courses than for upper-level courses. On the other hand, 
communicating orally with clients had a higher mean rating for the upper-level courses, which would 
include clinical courses. In the Listening category the rating of importance of identifying key lecture points 
was higher for first-year courses than for the upper-level courses surveyed, while there was no real 
difference between the ratings of the other three Listening tasks for the two levels of classes. This may 

there is more lecturing in traditional first-year courses than in the upper-level courses surveyed. 

In general, many more of the tasks listed in the survey were rated as more important for traditional 
first-year courses than for upper-level courses. The differences between the ratings of tasks for the two levels 
of courses are greatest for Analyzing Cases or Legal Problems, Reading, Reasoning, Writing, Organizing and 
Synthesizing, and Constructing Arguments. In all these cases the tasks were rated higher for first-year 
classes. Classroom observations confirmed that these are the fundamental tasks in the first-year classes, 
while upper-level courses seemed to involve a greater variety of tasks. There was also a considerable 
difference in the ratings for the two levels for Quantitative Reasoning tasks, where the ratings were higher 
for upper-level courses. This seems to reflect the presence of the Tax course as one of the upper-level courses. 

Even with these differences in ratings between the two levels, however, the ratings were convergent for 
the two levels of courses. Tasks that were rated as highly important for first-year courses were also rated 
highly important for upper-level courses and tasks not rated important for upper-level courses were also not 
rated as so important for first-year courses. In general, most of the tasks on the survey were rated more 
highly important for traditional first-year courses than for the upper-level courses surveyed. However, this 
may be because the upper-level courses surveyed were selected because they were taught differently from 
traditional first-year courses. 

Table 5 shows that, as expected, there are many significant differences between the first-year and 
upper-level courses. Only two task categories. Normative Thinking and Conducting Research, do not show 
significant mean differences between course levels. The t| values here are the largest of all the group 
comparisons, and five of them are larger than the .09 benchmark mentioned in the Data Analysis section 
above. These are the Organizing and Synthesizing tasks (r\ = 0.98), the Writing tasks (r) = 0.225), the 
Reasoning tasks (r) = 0.104), the Analyzing Cases or Legal Problems tasks (r) = 0.156), and the Reading tasks 
(q = 0.137). The Writing, Communicating Orally, Reading, and Analyzing Cases or Legal Problems task 
categories also have the largest values for the course-level comparisons, as can be seen in Table 6. This is 
consistent with the conclusions drawn above that tasks in these categories tend to be rated as more valuable 
in first-year courses overall. 

It is worth noting that the Writing task category is statistically significant across the sex, teaching status, 
and course level groupings, and it has the highest effect size value and the highest values of the 14 task 
groups for two of the three grouping. However, although there are differences for certain Writing tasks 
(such as writing briefs and summaries), the pattern of responses for the tasks remains consistent across the 
different respondent groupings. Tasks that are highly rated tend to be highly rated by all of the 
respondent groups. 
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Question: How do the ratings of importance of tasks differ between the individual courses surveyed? 

First-year Courses 

Figures 8a and 8b illustrate that the ratings of the importance of the 57 tasks were highly convergent for 
all of the traditional first-year courses surveyed except Legal Research and Writing. For the courses 
Contracts, Criminal Law, Property, Torts, Constitutional Law, and Civil Procedure the ratings are extremely 
close to each other. For Legal Research and Writing most of the Writing tasks. Working as Part of a Group or 
Interpersonally tasks. Constructing Arguments tasks, and Conducting Research tasks were rated as more 
important than for the other traditional first-year classes. In contrast. Listening tasks. Organizing and 
Synthesizing tasks, most Communicating Orally tasks, and writing exams were rated as less important for 
Legal Research and Writing. These results correspond with classroom observations of how traditional 
first-year courses are taught. Most traditional first-year courses are taught very similarly, organized around 
cases and using predominantly some combination of lecture and the Socratic method. Hence, they tend to 
involve the same tasks. Legal Research and Writing is organized and taught quite differently, with an 
emphasis on acquiring skills in legal analysis and communicating effectively in writing. So the similarity of 
ratings among most traditional first-year courses and the differences in ratings of tasks for Legal Research 
and Writing are what would be expected. 
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FIGURE 8a. Comparison of task mean ratings for all respondents for traditional first-year courses for the first 29 
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FIGURE 8b. Comparison of task mean ratings for all respondents for traditional first-year courses for the last 28 
tasks (N = 3,525) 
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Upper-level Courses 

Unlike traditional first-year courses, the upper-level courses do not show a convergent pattern of ratings 
of importance of tasks. For that reason the results are being displayed in tabular format, rather than as a 
graph, in Tables 7a and 7b. While the sample sizes for some of the courses are not very large, the table does 
offer some interesting insights. As one might expect, in the Professional Responsibility course. Normative 
Thinking tasks were rated as most important and Quantitative Reasoning tasks as least important. After 
Normative Thinking tasks. Problem Solving, Analyzing Cases or Legal Problems, Reasoning, and Reading 
were generally rated as most important. In contrast, respondents for the Tax course rated several of the 
Quantitative Reasoning tasks high and the Normative Thinking tasks lowest of any of the courses. However, 
respondents for the Tax course also rated Analyzing Cases or Legal Problems, Reading, and some Reasoning 
and Listening tasks high and rated Conducting Research, Working as Part of a Group or Interpersonally, and 
some Communicating Orally tasks very low. 

Respondents for Alternative Dispute Resolution courses rated Working as Part of a Group or Interpersonally 
and Listening tasks high and Analyzing Cases or Legal Problems, Quantitative Reasoning, and Reasoning tasks 
lower than most of the other courses. In contrast to all the other upper-level courses, respondents for the Clinical 
and Appellate courses rated Conducting Research tasks high. For the Clinical course nearly all the tasks were 
rated as quite important except for Quantitative Reasoning tasks and those, like reading class notes, outlining a 
course, and answering exam questions, that are more important in a traditional class structure. Reading tasks 
were also not rated as high as for traditional first-year courses. Respondents for the Appellate course rated 
Constructing Argument tasks most important, followed by Analyzing Cases or Legal Problems, Reasoning, 
Problem Solving, and some of the Reading and Writing tasks. They rated Quantitative Reasoning tasks lowest 
and, aside from orally arguing persuasively, did not rate Communicating Orally tasks as high. Listening tasks 
were also not rated as high as for other courses. All the tasks were generally not rated as high for the Seminar 
courses as for other courses. Rated highest were Constructing Argument tasks. Analyzing Cases or Legal 
Problems tasks. Problem Solving tasks. Normative Thinking tasks, and Listening tasks. 
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TABLE 7a 

Task mean ratings for all respondents for upper-level courses for the first 29 tasks 

Alternative 

Professional Dispute Clinical 

Tasks Responsibility Resolution Course 



Apellate 



Tax 



Other 

Seminar 



Managing Time 

Identifying academic goals 
Allocating time to tasks 
Working as Part of a Group 
Contributing to group 
Engaging in negotiation 
Constructing Arguments 
Identifying strongest point 
Organizing evidence 
Effectively countering 
Interpreting incomplete facts 
Listening 

Identifying key lecture points 
Distinguishing what was said 
Identifying what was implicitly said 
Raising appropriate questions 
Communicating Orally 

Answering basic class questions 
Answering other class questions 
Orally arguing persuasively 
Communicating with clients 
Organizing and Synthesizing 
Outlining a course for exams 
Identifying general law theories 
Formulating hypotheses 
Writing 

Writing class notes 
Writing briefs 
Writing exam answers 
Drafting legal documents 
Writing quickly 
Writing concisely 
Listing facts 

Arguing logically in writing 

Using proper grammar 

Editing 



2.76 


2.63 


3.02 


2.74 


2.82 


3.63 


2.20 


3.65 


2.65 


1.81 


3.94 


3.28 


3.01 


3.08 


3.65 


3.08 


3.05 


3.71 


2.92 


2.95 


3.53 


2.65 


2.74 


3.41 


3.36 


3.19 


2.66 


2.89 


3.29 


3.41 


2.80 


3.42 


3.45 


3.05 


3.46 


3.62 


2.90 


2.83 


2.38 


2.83 


2.74 


2.23 


2.75 


2.93 


3.35 


2.13 


3.02 


3.67 


2.56 


1.46 


1.38 


2.78 


2.09 


2.17 


2.43 


2.19 


2.66 


2.87 


2.29 


1.88 


1.89 


1.49 


2.57 


2.64 


1.62 


1.28 


1.53 


1.61 


3.65 


2.21 


1.65 


2.84 


2.84 


2.72 


3.61 


2.64 


2.58 


3.60 


2.96 


2.63 


3.57 


2.68 


2.90 


3.69 


2.32 


2.54 


3.54 



3.00 


2.96 


2.99 


3.33 


3.04 


3.09 


2.27 


1.86 


2.23 


1.82 


1.37 


1.90 


3.87 


2.59 


3.36 


3.86 


2.55 


3.30 


3.81 


2.34 


3.18 


3.56 


2.29 


3.01 


2.18 


3.51 


3.35 


2.56 


2.94 


3.18 


2.52 


2.66 


3.11 


3.13 


2.64 


3.22 


1.93 


2.87 


2.86 


1.93 


2.56 


2.62 


3.85 


2.08 


2.78 


1.71 


1.84 


2.00 


1.55 


3.24 


2.54 


2.35 


2.63 


2.73 


2.79 


2.25 


2.70 


2.04 


3.47 


2.77 


2.88 


1.88 


2.01 


1.40 


3.35 


2.51 


3.82 


1.40 


1.84 


2.29 


2.68 


2.60 


3.71 


3.06 


3.22 


3.48 


2.56 


2.87 


3.82 


2.67 


3.20 


3.81 


2.19 


3.00 


3.74 


1.76 


2.46 




36 



31 



TABLE 7b 

Task mean ratings for all respondents jor upper-level courses for the last 28 tasks 

Alternative 



Tasks 


Professional 

Responsibilitv 


Dispute 

Resolution 


Clinical 

Course 


Apellate 


Tax 


Other 

Seminar 


Reasoning 


Deducing a legal conclusion 


3.34 


2.31 


3.45 


3.60 


3.37 


3.15 


Generalizing case principles 


3.04 


2.15 


3.48 


3.67 


3.01 


3.08 


Identifying reasoning fallacies 


2.86 


2.54 


3.46 


3.49 


2.60 


3.06 


Comparing different 


2.87 


2.69 


3.33 


3.58 


2.61 


3.01 


reasonings 
Problem Solving 


Identifying legal problem 


3.45 


3.15 


3.82 


3.62 


3.46 


3.51 


Identifying legal alternatives 


3.01 


3.68 


3.80 


3.18 


2.84 


3.20 


Analyzing Cases or Legal Problems 


Identifying key legal facts 


3.07 


2.64 


3.87 


3.68 


3.21 


3.37 


Identifying legal basis 


2.96 


2.13 


3.48 


3.73 


3.08 


3.13 


Interpreting statutes 


3.22 


2.13 


3.80 


3.74 


3.64 


3.19 


Identifying rule of law 


3.37 


2.21 


3.69 


3.69 


3.58 


3.31 


Identifying legal arguments 


3.09 


2.97 


3.78 


3.76 


2.97 


3.33 


Identifying legal similarities 


3.01 


2.37 


3.66 


3.72 


3.01 


3.14 


Applying legal statutes 


3.43 


2.34 


3.68 


3.68 


3.57 


3.30 


Normative Thinking 


Identifying ethical issues 


3.66 


3.10 


3.32 


2.93 


2.34 


3.13 


Engaging in moral reasoning 


3.75 


3.16 


3.34 


3.07 


2.40 


3.24 


Comparing ethical principles 


3.68 


3.12 


3.16 


2.95 


2.23 


3.14 


Conducting Research 


Using law library 


2.02 


2.17 


3.52 


3.82 


1.53 


2.41 


Using electronic databases 


1.92 


2.10 


3.32 


3.85 


1.41 


2.27 


Reading 


Reading assigned material 


3.31 


3.03 


2.50 


2.69 


3.33 


3.32 


Reading class notes 


2.93 


1.74 


1.45 


1.87 


3.74 


2.66 


Reading supplemental mate- 


2.13 


2.03 


2.34 


2.79 


2.18 


2.43 


rial 


Reading large amounts 


2.30 


1.63 


2.16 


2.89 


2.35 


2.41 


Reading critically 


3.06 


2.70 


3.08 


2.89 


3.07 


3.07 


Quantitative Reasoning 


Making calculations 


1.40 


1.73 


1.64 


1.38 


3.31 


1.50 


Comparing quantities 


1.38 


1.69 


1.56 


1.36 


2.83 


1.49 


Interpreting formulas 


1.35 


1.52 


1.60 


1.35 


3.35 


1.43 


Interpreting statistics 


1.33 


1.57 


1.48 


1.41 


1.97 


1.50 


Interpreting graphs 


1.32 


1.54 


1.40 


1.40 


1.82 


1.44 



While there is no convergent pattern of the ratings of the importance of tasks across all upper-level 
courses, the differences among the courses make a great deal of sense when the subject matter and teaching 
methods of these courses are taken into account. And, while there are variations in which tasks are more 
important in which classes, the tasks that are rated as most highly important for the aggregate of courses 
also stand out as important for most of the upper-level courses. Where the upper-level courses differ is that 
for some courses, like Tax and Alternative Dispute Resolution, some tasks stand out as important that are 
not rated so important in most other courses. The upper-level courses in the survey were chosen because 
they are likely to differ from traditional first-year courses in subject matter and teaching methods. In the 
light of this, it is noteworthy that many of the tasks generally rated as highly important for all law school 
courses were also rated as highly important for these upper-level courses. 

Question: Do the ratings of importance of tasks for success in law school courses differ by race, ethnicity, 
primary language, or undergraduate major of students? 

Figures 9a and 9b compare the ratings of the importance of the tasks by respondents of different 
ethnicities — Native Americans, Asian Americans, African Americans, Hispanic Americans, Caucasians, and 
Other. There were relatively few respondents from some of these groups, so the results must be interpreted 
cautiously. While members of some ethnic groups rated all the tasks higher or lower than the members of 
other groups did, the pattern of responses was highly similar for all of the groups. The ratings by members 
of the Hispanic and Native American groups diverged the most from those of members of the other groups. 
However, the differences between the groups are small and the responses for all the groups strongly 
converge into the same pattern of relative significance. 
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FIGURE 9a. Comparison of task mean ratings by ethnic group for the first 29 tasks (N = 3,525) 
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FIGURE 9b. Comparison of task mean ratings tiy ethnic group for the last 28 tasks (N = 3,525) 
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A comparison was also made between the ratings of importance of tasks by respondents for whom 
English is not the primary language with those by respondents for whom English is the primary language, 
although that data is not presented. While those for whom English is not the primary language generally 
rated all tasks as more important than those for whom English is the primary language, there was no 
difference in the pattern of relative ratings for tasks. 

The self-reported undergraduate majors of students were categorized into seven groups: the arts, natural 
sciences and mathematics, technical, business-related, professional /education, humanities, and social 
sciences. There were essentially no differences in the importance ratings of tasks between these groups of 
respondents, so that data is also not presented. 

Question: To what extent does the list of tasks surveyed include all the tasks involved in success in the 
law school course surveyed? 

Question 5 on the survey asked respondents how exhaustive the list of 57 tasks was of the tasks involved 
in success in the course. The possible answers are: "include essentially all the tasks involved in success in the 
course," "include most of the tasks involved in success in the course but do not include some other tasks 
involved in success in the course," "include some of the tasks involved in success in the course but do not 
include many other tasks involved in success in the course," and "include few, if any, of the tasks involved in 
success in the course." In Figure 10 the bars represent the responses of female faculty, male faculty, female 
students, and male students. The different shaded sections of each bar represent the percentage of 
respondents giving each answer. In all four cases, over 50% of the respondents indicated that the list of tasks 
includes essentially all the tasks necessary for success in the course. In fact, over 80% of participants 
responded with either "includes essentially all the tasks ..." or "includes most of the tasks." This indicates 
that by far most respondents regarded the list of tasks as very exhaustive of the tasks involved in success in 
the course. 
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^/Ssl fN =T/ 76 j° ” female faculty and students to the question of how complete the list 
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There were, though some differences in the responses to Question 5 between faculty and students and 
be^een male and female faculty members. In general faculty members considered the list to be more 
^haustive than students did. The responses of male and female students are strikingly similar to each other 
However, over 70 /o of male faculty members considered the list to include essentially all the tasks required ' 
for the course whi e just over 60% of the female faculty did. Over 90% of the male faculty responded either 
includes essentially all the tasks ..." or "includes most of the tasks" compared to just under 90% of the 
female faculty. Nevertheless, the list was regarded as highly inclusive by nearly all (80%) of the respondents 
which suggests that it can be regarded as a reasonably comprehensive list of the tasks important to success in 
the law school courses surveyed. 

Were the responses to the question of how exhaustive the list of tasks is different for different courses? In 
Figure 11 the bars represent either law school levels (i.e., first-year or upper-level) or individual courses. Bars 
1 and 9 represent the two levels. Bars 2-8 represent first-year courses and bars 10-15 represent upper-level 
courses. The differently shaded sections of the bars represent the percentage of respondents giving each 
answer. A higher percentage of respondents judged the list of tasks to contain essentially all the tasks 
invo ved in success in traditional first-year courses than judged the list to contain essentially all the tasks 
involved in success in upper-level courses. Around 60% of the respondents for traditional first-year courses 
judged ^ tasks to contain essentially all the tasks involved in success in the course, compared to 

aroun 50 /o of the respondents for upper-level courses. The percentage responding ''contains essentially all" 
did not vary much from 60 /o for any of the first-year courses individually. The percentages answering 
contains essentially all" for the upper-level courses were all below 50% except for Tax. For Professional 
Responsibility, Alternative Dispute Resolution, and a Clinical course, the percentages were closer to 40% 
When the percentage of respondents answering "contains most of the tasks" are added to the percentage 
answering essentially all", the responses for first-year courses fall between 85% and 90%. The percentage 
giving these answers for upper-level courses is both lower and more variable. For upper-level courses taken 

togethe^ the percentage is just under 80% and it ranges from 80% for a Clinical course. Tax, and a Seminar to 
under 70% for Alternative Dispute Resolution. ^cmuici lu 
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FIGURE 11. Proportion of responses for each course to the question of how complete the list of tasks is (N = 3,326) 
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Thus, the list of tasks in question 4 was considered very comprehensive and complete for traditional 
first-year courses and somewhat less so for upper-level courses. However, even in the case of upper-level 
courses selected as likely to be taught differently from the ways traditional first-year courses are taught, the 
percentage of respondents answering that the list of tasks contains essentially all or most of the tasks 
involved in success in the course is very high. This would seem to indicate that the list includes most ox the 
tasks involved in most law school courses at the schools surveyed, both in traditional first-year courses and 
upper-level courses. And this suggests that the list of tasks in the survey provides a good basis for 
determining what skills are essential for success in law school. 

Since some respondents indicated that the list of tasks in question 4 did not include essentially all the 
tasks involved in success in the course, question 6 on the survey offered respondents the opportunity to offer 
additional tasks not mentioned in question 4. Of the respondents, 664 (19%) gave a response to this question. 
Many of the respondents simply repeated tasks mentioned in question 4, such as listening, writing, public 
speaking, organization, time management, research, and problem solving. However, some different tasks 
were also mentioned. Most widely mentioned across courses was memorization. Also mentioned by 
respondents for several courses were tasks related to study groups and preparation for examinations. And 
for several courses, most notably Alternative Dispute Resolution and Clinical, personal communication and 
negotiation tasks were mentioned. 

Question: Do the tasks involved in success in law school courses vary according to how the courses 
are taught? 

Hoiv Were the Courses Surveyed Taught? 

Discussions with law school faculty and students and observation of law school classes suggest that 
courses may be taught in different ways. The "traditional" method for teaching a law school course is case 
study and the use of the Socratic method in class, with students responding to faculty questions about cases. 
In practice, the Socratic method is often supplemented by a certain amount of lecturing. However, other 
teaching methods are also employed. Many courses use a problem solving method in which a hypothetical 
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legal problem is posed and students have to work out a solution, given the law as they understand it Other 
meth^ods include role-playing, seminars, clinical work, and group projects. ^ 'understand it. Other 

o answer the question of how the tasks involved in success in law school are affected by the way 
courses are taught, we first asked how the courses being surveyed were taught Because of the historkal 

st^'^y/Socratic method approach to teaching law schLl courses, the crucial difference 
methods was se^ to be between this traditional method and all other methods together For that 
eason the survey asked in Question 2 which of the responses "best describes the teaching techniques or 
methods used in the course" and the responses were: (1) "Case study using exclusively leLre Ve Socratic 
Method, or a combination of the two," (2) "Case study using other techniqLs or methods (e g smaU ^roi^^ 
^erases) m addition to lecture or the Socratic Method," (3) "Case study not using lecture orfhe Socratic ^ 
Method but instead using other techniques or methods (e.g., group exercises, open discussion) " and (4) "A 

^ g'viug the proportion of respondents indicating that a particular teaching method 

hA/n ^ course or for first-year and upper-level courses taken together. Bars 1 and 9 represent the 

Tff levels^ Bars 2-8 represent first-year courses and bars 10-15 represent upper-level courses. The 

ifferently shaded sections of each bar on the graph indicate the percentage of respondents giving each response. 

FIGURE 12. Proportion of teaching methods used in each course (N = 3,315) 
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All the traditional first-year courses but one were described as predominantly taught using case studv 
P “[both. The only exception to this predominant response pattern is ^ 

g Research and Writing for which over 60% of the responses indicate some method other than case 
ove^ 60“/' tlf over 75 /o indicate some techniques other than lecture or Socratic Method. As a result, 
over 60 A of the responses for all of the courses identified as traditional first-year courses (top-most barl 

I'eclure and^th Socratic Method exclusively and over 80% indicate c^ase study with 

Socratic Method either exclusively or in conjunction with other methods or technique! These 
nrpHn seem to indicate that case study and the use of lecture and/or the Socratic Method are the 

predominant ways of teachmg the traditional first-year courses in law school. 

.the'rVba';Sld ^or the upper-level courses taken 



together (bar labeled "upper level"), less tha^n" 30% of the responsesTdicated 
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and the Socratic Method was used exclusively. These methods were indicated predominantly for three 
courses, Tax, a Seminar, and Professional Responsibility, ranging from around 30% for Professional 
Responsibility to around 45% for Tax. For Alternative Dispute Resolution, Clinical courses, and Appellate 
courses, less than 5% indicated this teaching method. Between 80% and 90% of the respondents for both the 
Clinical courses and the Appellate courses indicated some other method than case study and over 50% of the 
respondents for Alternative Dispute Resolution indicated a method other than case study. These responses 
are not surprising, however, since the upper-level courses surveyed were chosen because they were judged 
likely to use techniques other than those used in traditional first-year or core courses. As expected, the 
results indicate that indeed some upper-level courses are primarily taught using techniques other than case 
study with lecture and the Socratic Method. 

The comparison of responses by faculty and students and by males and females regarding methods and 
techniques shows a clear difference between the responses of male and female faculty about the teaching 
methods or techniques they use (Figure 13). Over 70% of the male faculty indicated that they used case study 
with lecture and Socratic Method either exclusively or in conjunction with other methods. However, only 
between 50% and 55% of the female faculty gave this response. Indeed, less than 25% of the female faculty 
indicated that they used case study with lecture and Socratic Method exclusively while nearly 50% of the 
male faculty indicated that they did. Further, more than 45% of the female faculty indicated that they used 
some other method than case study, while less than 25% of the male faculty did. Figure 5 indicates that there 
is a difference in which courses male and female faculty responded to the survey for and this difference may 
partially account for the difference in teaching methods used. 



FIGURE 13. Proportion of teaching methods reported by male and female faculty and students (N - 2,935) 
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Mal6 and female students, on the other hand, gave very similar responses regarding the teaching 
methods and techniques used in courses. ° 

Question 3 asked respondents to briefly describe any teaching techniques or methods used in the course 
other than lecture or Socratic Method or case study. This question was addressed by 1,610 respondents. 
Methods most commonly mentioned were group exercises, student presentations, problem solving, use of 
hypotheticals, open discussion, debate, and writing. For some courses— particularly Alternative Dispute 
Resolution role-playing was mentioned. For a few courses, working with clients was mentioned Some 
specific courses also mentioned use of films and videos. It is noteworthy that many of these methods involve 
categories of tasks and skills not ranked so high in Question 4— Working as Part of a Group or 
Interpersonally and Communicating Orally. ^ 

How Are the Task Ratings Related to Methods of Teaching? 

Figures 14a and 14b compare the task means for courses using each of the four teaching methods 
menhoned m the survey. For the tasks associated with Managing Time, Constructing Arguments, Problem 
bolvmg, and Quantitative Reasoning, there was no apparent pattern of difference in the ratings for different 
methods of teaching. Managing Time seems to be rated moderately important for all the teaching methods 
Tasks involved m Constructing Arguments and Problem Solving were rated as important for all the methods 
of teaching. In contrast, tasks involving Quantitative Reasoning were rated as not very important. 



FIGURE 14a. Comparison of task means for all respondents by teaching method for the first 29 tasks (N = 3,525) 
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Tasks in the categories Reasoning, Analyzing Cases or Legal Problems, Listening, Organizing and 
Synthesizing, and some of the Writing tasks were rated as somewhat more important for courses involving 
lecture and the Socratic Method either exclusively or with other methods added. The differences were not 
great, however, and the Reasoning and Analyzing Cases or Legal Problems tasks were rated as quite 
important for all methods of teaching. Listening and Organizing and Synthesizing were rated as somewhat 
less important for all methods. The category of tasks with the biggest difference between those courses 
including lecture and the Socratic Method and those not, was Organizing and Synthesizing. 

The Communicating Orally tasks involving answering questions in class were rated as more highly 
■Q“ortant in case study courses than in courses not using case study, whereas arguing persuasively was 
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rated at about the same importance for courses using all the methods. Communicating orally with a client 
was rated higher for course methods other than case study, indicating its relation to clinical courses. 
However, none of the Communicating Orally tasks except answering class questions was rated as highly 
important for any of the methods. 



FIGURE 14b. Comparison of task means for all respondents by teacing method for the last 28 tasks (N = 3,525) 
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Many of the Writing tasks were rated as quite important for all teaching methods, particularly arguing 
logically and writing concisely. 

Conducting Research tasks were least important for courses using case study with lecture and the 
Socratic Method exclusively and most important for courses not using case study. Reading tasks showed just 
the opposite tendency, being rated most important in case study classes using lecture and the Socratic 
Method, and least important for courses not using case study. However, Conducting Research tasks were 
generally not rated highly important, while certain Reading tasks were rated highly important for all 
teaching methods. 

Normative Thinking tasks were rated most highly important for case study courses including methods 
other that lecture and Socratic Method and somewhat less important for courses using lecture and Socratic 
Method exclusively and for those not using case study. However, Normative Thinking tasks were rated 
moderately important for all methods of teaching. 

Tasks involved in Working as Part of a Group or Interpersonally were rated rather low in importance for 
courses using case study with lecture and the Socratic Method exclusively, and of somewhat higher 
importance for courses using methods other than lecture and the Socratic Method. This would suggest that 
at least some of these methods involved more group interaction than did lecture or the Socratic Method. 

The general pattern of importance of certain types of tasks for success in law school courses holds, with 
a few exceptions, regardless of the teaching method employed in the course. Certain tasks in the categories 
of Analysis of Cases, Reasoning, Reading, Problem Solving, Constructing Arguments, Listening, and Writing 
are generally ranked more important regardless of the teaching method used in a course and other tasks are 
not rated as important regardless of the teaching method used. 
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Question: Do the judgments of completeness of the list of tasks vary according to the teaching method 
used in a class? 

Since courses using methods other than case study with lecture and the Socratic Method might involve a 
greater variety of tasks than those using a more limited range of methods, one could expect that the list of 57 
tasks in the survey might be less likely to be regarded as including essentially all or most of the tasks 
essential for success in those courses involving methods other than lecture and the Socratic Method. And to 
a certain extent that is the case. In Figure 15, each bar represents one of the four teaching methods in the 
survey and the different shaded areas indicate the proportion of respondents answering that the list of 57 
tasks includes essentially all, most, some, or few if any of the tasks essential to success in the course, 
respectively. The proportion of responses for all courses including lecture and the Socratic method, either 
exclusively or along with other methods, are nearly identical. A little over 60% of the respondents indicated 
that the list included essentially all the tasks and nearly 90% indicated that the course included essentially all 
or most. However, the proportion of responses for courses not including lecture and the Socratic method was 
not greatly different. Around 50% of the respondents for courses without lecture or the Socratic Method 
responded that the list included essentially all the tasks and nearly 80% responded essentially all or most. So, 
while the list of 57 tasks appear somewhat less exhaustive of the tasks essential for success in courses not 
involving lecture or the Socratic Method, it was still judged to contain most of the tasks involved in success 
in most courses regardless of the teaching method used. 



FIGURE 15. Proportion of responses for each teaching method to the question of how complete the list of tasks is 
(N = 3,285) 
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Conclusions 



The basic question that the Skills Survey sets out to answer is, "what tasks are fundamental to success in 
law schoo courses. It does so by sampling the judgments of students and faculty about the importance of 
certain tasks m parhcular law school courses. While the sample of respondents is not exactly representative 
of the total population of law school students and faculty, it includes a very large number of respondents 
from a large number of schools and is quite similar to that population in most respects. Moreover the 
responses strongly converge into a clear pattern that varies little among most subgroups in the population of 
respondents or between courses surveyed. The most significant differences in ratings of importance of tasks are 
between faculty and students and between first-year courses and upper-level courses. However, these 
differences do not significantly change the pattern of relative ratings of responses. Thus, we believe the survey 
provides a meaningful picture of what tasks are important for success in the law school courses surveyed. 

The survey responses display a pattern with the categories of tasks in the survey falling into four tiers of 
importance determined by the mean task ratings and the percentage of respondents rating the tasks highly 
or moderately important. The first tier includes the category of tasks generally rated as most important to 
success in most law school courses— tasks involved in Analyzing Legal Cases and Problems. While the skills 
directly applicable to these tasks are taught in law school, there are certain fundamental skills of critical 
reading, analysis, and reasoning that underlie these tasks and that the current LSAT seeks to assess. 

The categories of tasks in the second tier were also judged very important for success in most law school 
courses. These include Reasoning, Reading, Problem Solving, Constructing Arguments, Writing, Listening, 
and Managing Time tasks. The tasks in these groups generally were rated as important or highly important 
y all groups of respondents for nearly all courses. It is noteworthy that the tasks involved in the first four of 
the categories Reasoning, Reading, Problem Solving, and Constructing Arguments — predominantly 
involve skills of critical reading and analysis, reasoning, problem solving, and argument analysis that the 
current LSAT seeks to assess. However, the skills for two other categories of important tasks in this tier are not 
tested by the current LSAT— Listening and Writing. The results of the survey suggest that the skills involved in 
these latter tasks are also fundamental to success in law school and that LSAC should investigate ways of 
testing them on the LSAT. VVriting and Listening tasks are not only generally important in law school, they are 
also particularly important in some upper-level law school courses aimed at legal practice skills. The final 
category in this tier is tasks involved in Managing Time. These tasks involve skills in long-term time 
management, which there is reason to think cannot be assessed in a single-day test like the LSAT. The LSAT 
does, however, require short-term time management in answering questions under timed conditions. 

The third tier includes tasks that are also important for success in law school, but not to the extent of 
those in the higher tiers or not so generally. The categories of tasks in this tier include Organizing and 
Synthesizing, Communicating Orally, and Normative Thinking. Communicating Orally tasks are ranked as 
generally less important than Writing tasks, even though they are important in classroom participation and 
in clinical and legal practice courses. The skills involved in these tasks are not currently assessed on the 
LSAT. Organizing and synthesizing tasks are regarded as more important by students than by faculty and 
are generally regarded as more important in traditional first-year courses in which students are required to 
discover the patterns of common law. The skills involved in these tasks are tested to a degree on the current 
LSAT in all three item types. Normative (ethical and public policy) Thinking tasks are rated as much more 
important by faculty than by students, but they generally are rated as rather important across the board in all 
courses and by all respondents. The present LSAT includes arguments and passages about ethical and public 
policy topics, although it does not include specific questions designed to measure normative thinking skills 

The survey results suggest that the tasks in the categories of Conducting Research, Quantitative 
Reasonmg, and Working in Groups are not regarded as important in most law school courses. There are some 
courses, however, in which these tasks do play a more important role, such as Alternative Dispute Resolution, 
Appellate, and Tax. The skills involved in these categories of tasks are not currently assessed on the LSAT. 

Finally, the four-tier pattern of importance basically holds for all respondents and for nearly all courses, 
except for some differences between faculty and students. Three categories of tasks were rated at a higher 
level by faculty than by students. These were Constructing Arguments, Reasoning, and Normative Thinking. 
The faculty rahngs place Constructing Arguments and Reasoning in the first tier rather than the second, and 
Normative Thinking in the second tier, rather than the third. However, despite these differences, the overall 
pattern of relative importance of the tasks and the general levels of importance do not differ greatly between 
faculty and students. There are virtually no practical differences in the responses of male and female 
students, and the small differences in the responses of male and female faculty may be a product of the 
differences in the courses for which they responded. While there also were very small differences in the 
degree to which tasks were rated as important between respondents of different ethnicity, and between those 
whose primary language is other than English and those whose primary language is English, the pattern of 
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rebtive importance of the tasks and the general level of importance of the tasks was basically the same for all 
ethnic and language groups. ^ 

There were differences in the levels of importance of tasks and the patterns of relative importance 
between traditional first-year courses and upper-level courses. The data suggests that this may be due 
primarily to differences in the way in which these courses are taught. However, even taking these variations 
into account, most first- and second-tier tasks and some third-tier tasks were still rated as quite important for 
nearly all courses and teaching methods. This suggests that they are generally fundamental to success in law 
school courses. 

Among respondents taken as a whole, the list of tasks on the survey was judged to contain essentially all 
or most of the tasks involved in success in the course by over 80% of the respondents. The list was judged as 
more exhaustive of tasks fundamental to success in traditional first-year courses. The responses to the survey 
and to the open-ended questions suggest that additional academic tasks may be involved in upper-level 
courses taught differently from traditional first-year classes. However, these additional tasks would appear 
o be course specific and not as important for success in law school in general as tasks rated highly important 
tor all or most law school courses. The tasks rated as most important for traditional first-year courses were 
also predominantly rated as important for the upper-level courses surveyed. 
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LSAC LAW SCHOOL COURSE TASK SURVEY 



Law School: 



Law School Course: 



Instructions: The following survey is designed to collect information about the tasks required of 
students by the above-named law school course only. If you have not taken or taught the above- 
named course, please cross out the name and, if it is a first-year course, write in another first-year 
course you have taken or taught. If it is an upper-level course, write in the name of another 
similar upper-level course you have taken or taught. Please answer each question either by 
checking the appropriate box or by writing a brief response in the space given. 

1 . Please check the category that applies to you. 

□ Law school teacher of the above-named course. 

□ Law school student who has taken the above-named course. 

2. Based on your experience with the above-named course, which one of the following best 
describes the teaching techniques or methods used in the course? (Please check only one 
response.) 

□ Case study using exclusively lecture, the Socratic Method, or a combination of the two. 

□ Case study using other techniques or methods (e.g., small group exercises) in addition to 
lecture or the Socratic Method. 

□ Case study not using lecture or the Socratic Method but instead using other techniques or 
methods (e.g., group exercises, open discussion). 

□ A teaching approach other than case study (e.g., historical survey, problem solving, focus 
on certain academic or legal skills, clinical practice). 

3. Briefly describe any teaching techniques or methods used in the course in addition to or other 
than lectures or the Socratic Method. In addition, if case study was not the primary teaching 
method used in the course, please briefly describe the method used. 




50 



4. Below is a list of tasks that might be involved in taking a law school course. Based on your 
experience with the above-named law school course, please rate each of the following tasks 
in terms of its importance to success in the course by checking the appropriate box to the 
right of the task. (Please check one and only one box per task. The order in which the tasks 
are presented is not intended to be an indication of their importance.) 





Category: Managing Time 


HIGHLY 


MODERATELY 


SOMEWTIAT 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 










NOT 












APPLICABLE 


1. 


Identifying academic goals and priorities and 
the tasks necessary to achieve those goals 


□ 


□ 


□ 


□ 


2. 


Allocating available time to tasks based on 
priorities 


□ 


□ 


□ 


□ 


Category': Working as Part of a Group or 


HIGHLY 


MODERATELY 


SOME\nUT 


NOT 




Interpersonally 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 

NOT 




TASK 








APPLICABLE 


3. 


Contributing to a group assignment, a 


□ 


n 


□ 


□ 




discussion group for a class, or group exam 
preparation 


LJ 








4. 


Engaging in negotiation, conflict resolution, 
or arbitration, including identifying issues in 
dispute, identifying common ground among 
disputing parties, and identifying solutions or 
resolutions on which parties can be brought 


□ 


□ 


□ 


□ 




to agree 










Category: Constructing Arguments 


HIGHLY 


MODERATELY 


SOMEWHAT 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 










NOT 












APPLICABLE 


5. 


Identifying the strongest evidence or reasons 
for or against a position 


□ 


□ 


□ 


□ 


6. 


Organizing evidence and reasons into a 
logically coherent argument for one’s 
position 


□ 


□ 


□ 


□ 










7. 


Effectively countering or rebutting the 
evidence or reasons against one’s position 


□ 


□ 


□ 


□ 


8. 


Interpreting or shading incomplete or 


□ 


n 


n 


n 




inconclusive facts to support a conclusion 


LJ 


LJ 


LJ 




Category: Listening 


inoiiLV 


MODERATEI.'S' 


somewhat 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 










NOT 












APPLICABLE 


9. 


Identifying the key points in lectures and 


□ 


n 


□ 


□ 




class discussions 


1 1 


10. 


Distinguishing precisely wliat a person has 
explicitly said and not said 


□ 


□ 


□ 


□ 


11. 


Identifying what is implicit in what a person 
has said 


□ 


□ 


□ 


□ 


12. 


Raising appropriate questions and arguments 


n 


n 


□ 


□ 




in response to what others have said 


LJ 


LJ 
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Category: Communicating Orally 


HIGHLY 


MODERATELY 


SOMEWTIAT 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 












NOT 


13. 










APPLICABLE 


Answering in class basic questions about the 


n 


n 








facts or analysis of an assigned reading 


LJ 


LJ 


U 


□ 


14. 


Answering in class questions about 


n 


1 — 1 








unassigned hypothetical cases 


U 


U 


□ 


□ 


15. 


Orally arguing persuasively for a position 


□ 


□ 


□ 


□ 


16. 


Communicating orally with a client 


□ 


□ 


□ 


□ 


Category: Organizing and Synthesizing 


HIGHLY 


MODERATELY 


SOME)MIAT 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 












NOT 


17. 


Outlining a course for fmal exams, including 








APPLICABLE 




identifying key cases and their facts, holdings, 
and principles 


□ 


□ 


□ 


□ 


18. 


Identifying and assessing general theories that 
unify diverse areas of law 


□ 


□ 


□ 


□ 


19. 


Formulating effective hypotheses to account for 
incomplete, inconclusive, or apparently 
incompatible information 


□ 


□ 


□ 


□ 




Category: Writing 


HIGHLY 


MODERATELY 


SOMEWHAT 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 












NOT 


20. 


Writing class notes 








APPLICABLE 




□ 


□ 


□ 


□ 


21. 


Writing briefs and summaries of cases and legal 
problems 


□ 


□ 


□ 


□ 


22. 


Writing answers to exam questions 


□ 


□ 


□ 


□ 


23. 


Drafting legal documents: e.g., case or appellate 
briefs, memoranda, complaints, responses to 
complaints, letters to clients or to other counsel, 
and legal instruments such as contracts or wills 


□ 


□ 


□ 


□ 


24. 


Writing quickly and fluently under time pressure 


□ 


□ 


□ 


□ 


25. 


Writing very concisely and with clarity 


□ 


□ 


□ 


□ 


26. 


Listing or describing facts or information in 
writing 


□ 


□ 


□ 


□ 


27. 


Arguing logically and persuasively in writing 


□ 


□ 


□ 


□ 


28. 


Using proper grammar, punctuation, and spelling 


□ 


□ 


□ 


□ 


29. 


Editing or rewriting 


□ 


□ 


□ 


□ 
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Category: Reasoning 


lilGHLY 


MODERATELY 


SOMEMtTUT 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


IMPORTANT 


lmportant/ 












NOT 


30. 


Deducing a conclusion (decision) from a legal 








APPLICABLE 




rule, principle, or statute and a set of facts 
[deduction] 


□ 


□ 


□ 


□ 


31. 


Generalizing or synthesizing principles or rules 

from relevantly similar cases 

[induction] 


□ 


□ 


□ 


□ 


32. 


Identifying contradictions and fallacious or 
Hawed reasoning 


□ 


□ 


□ 


□ 


33. 


Identifying and comparing patterns of reasoning 
in difi'erent jirguments 


□ 


□ 


□ 


□ 




Category: Problem Solving 


HIGHLY 


MODERATELY 


SOMEVtllAT 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 












NOT 












APPLICABLE 


34. 


Identifying a legal problem and the legal issues 
involved in that problem 


□ 


□ 


□ 


□ 


35. 


Identifying alternatives and developing and 
implementing a plan of action based on identified 
goals and objectives 


□ 


□ 


□ 


□ 




Category: Analyzing Cases or 


HIGHL'^' 


MODERATELY 


SOMEMtTIAT 


NOT 




Legal Problems 


IMPORTANT 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 




TASK 








NOT 

A PPT IPART P 


36. 


Identifying the key facts in a case or legal 








y\i 1 




problem and the legal issues involved 


□ 


□ 


□ 


□ 


37. 


Identifying the basis for a legal decision 
(ratio decidendi) 


□ 


□ 


□ 


□ 


38. 


Interpreting statutes or opinions in relation to a 
case or legal problem 


□ 


□ 


□ 


□ 


39. 


Identifying the principle, rule of law. or statute 
operating in or applicable to a case or a legal 
problem 


□ 


□ 


□ 


□ 


40. 


Identifying and evaluating tlic arguments, claims. 


n 


n 








assumptions, or positions in a case or a legal 
problem 


LJ 


U 


□ 


□ 


41. 


Identifying the relevant similarities or difi'erences 
of a case or a legal problem with other cases or 
legal problems 


□ 


□ 


□ 


□ 


42. 


Applying a case. rule, principle, or statute to a 
new or h>potlictical set of facts 


□ 


□ 


□ 


□ 
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Category : Normative Thinking 


HIGHLY 


MODERATELY 


SO.MEWHAT 


NOT 




TASK 


important 


important 


IMPORTANT 


IMPORTANT/ 












NOT 


43, 


Idcnii lying eiliical and public policy issues and 








APPLICABLE 




distinguishing them from non -ethical or non- 
policy issues or mere matters of fact 


□ 


□ 


□ 


□ 


44. 


Engaging in reasoning involving public policy or 
moral or ethical principles 


□ 


□ 


□ 


□ 


45. 


Comparing the weight of competing ethical 
principles or public policv considerations 


□ 


□ 


□ 


□ 




Category: Conducting Research 


HIGHLY 


MODERATELY 


SOMEt^TlAT 


NOT 




TASK 


important 


IMPORTANT 


IMPORTANT 


IMPORTANT/ 












NOT 


46. 


Using the law library and legal references to 








APPLICABLE 




locate relevant sources of law 


□ 


□ 


□ 


□ 


47. 


Using electronic databases such as LEXIS and 
Wcsil..aw 


□ 


□ 


□ 


□ 




Category: Reading 


HIGHLY 


-MODERATELY 


SO-MEVYHAT 


NOT 




TASK 


IMPORTANT 


IMPORTANT 


important 


important/ 












NOT 


48. 


Reading a.ssigncd materials in casebooks. 








APPLICABLE 




textbooks, or reserved readings 


□ 


□ 


□ 


□ 


49. 


Reading class notes, outlines, and sample exam 
questions to prepare for the final 


□ 


□ 


□ 


□ 


50. 


Reading supplemental materials: e.g., legal 
treatises, legal encyclopedias, law reviews, hom 


□ 


□ 


□ 


□ 




books, etc. 










51. 


Reading large amounts of material under time 












constraints 


□ 


□ 


□ 


□ 


52. 


Reading critically and interactively, asking 
questions, tiikinu notes, tind rereading 


□ 


□ 


□ 


□ 


Category; Quantitative Reasoning 


HIGHLY 


-MODERATELY 


SO.MEWHAT 


NOT 




TASK 


IMPORTANT 


LMPORTA-NT 


IMI^ORIA-NT 


important/ 












NOT 


53. 


Making calculations 








APPLICABLE 






□ 


□ 


□ 


□ 


54. 


Comparing quantities 


□ 


□ 


□ 


□ 


55. 


Interpreting and applying formulas 


□ 


□ 


□ 


□ 


56. 


Interpreting statistics 


□ 


□ 


□ 


□ 


57. 


Interpreting basic graphical representations 


□ 


□ 


□ 


□ 
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5. The tasks listed in question 4 (please check only one response.) 

□ include essentially all the tasks involved in success in the course. 

□ include most of the tasks involved in success in the course but do not include some other 
tasks involved in success in the course. 

□ include some of the tasks involved in success in the course but do not include many other 
tasks involved in success in the course. 

□ include few, if any, of the tasks involved in success in the course. 

6. In the left-hand column of boxes write any tasks not listed in question 4 (one per box) that 
you believe, based on your experience with this course, are involved in success in this course. 
Please rate each of these additional tasks in terms of its importance to the course by checking 
the appropriate box to the right of the task w'ritten in. (Please check only one box per task.) 



Tasks 


Highly 

Important 


Moderately 

Important 


SOMEV^TlAT 

Important 



































7. Please add any further comments you may have regarding the tasks required for the above- 
named course. If necessary use the back of this page. 
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Demographic Information 

(For statistical purposes only.) 



D I . Are you: □ Male? □ Female? 

D2. How would you describe yourself using the following terms? (Check only one box.) 

□ Aboriginal Indian/American Indian/Alaskan Native 

□ Asian/Asian American/Pacific Islander 

□ Black/African American (Not of Hispanic origin) 

□ Latino(a)/Chicano(a)/Puerto Rican/or other Hispanic 

□ White/Caucasian (not of Hispanic origin) 

□ Other (state) 

D3. What year were you bom? 19 

D4. What is your dominant Iang;uage? □ English □ Other (state) 



If you were a student during the 1996-97 academic year: 



D5. Which of the following courses did you take during your first year of law school? 



□ Civil Procedure 

□ Contracts 

□ Property 

□ Criminal Law 

□ Introduction to Tax 



□ Torts 

□ Legal Research and Writing 

□ Constitutional Law 

□ Moot Court 

□ Introduction to International Law 



D6. Please list below any other courses that you have taken during your first-year of law school: 



D7. What was your approximate first-year law school GPA? 

GPA Scale 

D8. What was your undergraduate major? 

If you were a faculty member during the 1996-97 school year: 
D9. Please list below any other courses that you have taught: 

DIO. From what school did you receive your law degree? 
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Appendix B 



Abbreviations for Task Categories Used in the Figures 



MT 


Managing Time 


GR 


Working as Part of a Group or Interpersonally 


AR 


Constructing Arguments 


LI 


Listening 


OR 


Communicating Orally 


OS 


Organizing and Synthesizing 


WR 


Writing 


RS 


Reasoning 


PS 


Problem Solving 


AC 


Analyzing Cases or Legal Problems 


NT 


Normative Thinking 


CR 


Conducting Research 


RD 


Reading 


QR 


Quantitative Reasoning 
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